The Basics Of Occlusion

Occlusion is a powerful feature in Mental Ray (Mental Ray is a renderer that comes with Maya as a plug-in). It allows you to simulate perfect lighting without creating a light in your scene.

Mental Ray Plug-in

Mental Ray is shipped out with all versions of Maya. To make sure you have your plug-in loaded, go to window>settings/preferences>plug-ins manager. Check Load and Auto Load for the Mayatomr.mll plug-in. By checking Auto Load you won't have to do this step again because Mental Ray will be automatically loaded when you close and reopen Maya.

Setting Up A Simple Occlusion

Step one - Download the scene occlusion_setup.mb and open it in Maya.

Step two - You'll notice that all the objects in the scene are assigned with a standard Lambert shader. For occlusion to work, you connect an occlusion texture node to the ambiance attribute of the shader (if you don't know how to access the shader, select the model and hit the wavy checkerboard icon on the shelf; this will open the attributes editor with the shader selected).

step three - Now click the checkerboard icon beside then ambiance attribute; this will open the create render node window. When you create a node here, its output will be connected to the ambiance attribute.

Step four - Now go to the Mental Ray tab and roll down until you find the section textures. Create the Mib_amb_occlusion node. Now the occlusion has been set up.

Step five - If you were to render right now you wouldn't see anything. This is because this is a Mental Ray texture and you're rendering with Maya software by default. To fix this, open the Render Settings window by going to window>render editors>render settings. Set the Render Using to Mental Ray. Now when you render you'll be able to see some very sketchy quality rendering. We'll explain setting up the occlusion node in the section below.

Setting Up The Occlusion Node

Now, if the attributes editor isn't already opened up to the occlusion attributes, select the model and open up its shaders attributes and click on the new icon that has replaced the checkerboard icon beside the ambiance attribute. This will take you to the occlusions attributes.

We'll fix this rendering step by step.

Step one - First we've got to get rid of this "noisy effect". This is because the sampling is too low. The higher the sampling there will be less noise in the rendering. Be careful when increasing this number because it also increases render time. Set the samples to 70; this should get you a nice clean rendering.

Step two - The next attributes are Bright and Dark. The bright attribute is the color that will appear in the bright areas, and the dark attribute is the color that will appear in the dark attributes. Leave these attributes at their defaults because that way it maintains the same ratio. If your image is too light or you just want to make it darker, lower the value of the bright attribute; if you were to increase the dark attribute, the image will start looking flat.

Step three - Now for the spread attribute. The higher this attribute the more shading your model will have. In this case, set it to 1.

Step four - The next attribute is a check box for reflective. This means that the occlusion will be set up to render for a shader that has a high reflection. In this case we can leave this blank because we're not rendering a reflective fish.

The attributes below aren't necessary to change and will not be covered in this tutorial. They will be covered in a different Mental Ray tutorial because this tutorials main focus is setting up occlusion.

Using Occlusion For Layered Compositing

This is a very common thing to do. You render one image with only the color (no shading or anything else), then you render an occlusion image, and after that you lay the two images on top of each other, and then use the multiply filter to blend the two layers.

For this part of the tutorial, we'll be using a Mahi Mahi model (you can buy this model on TurboSquid).

Step one - Select your model and assign a surface shader to it. You can find this on the rendering shelf. A surface shader doesn't add shading to the model.

Step two - In the Out Color attribute, click on the checkerboard and create a file node. On the image name attribute, load the texture by clicking in the folder icon and find the texture on your hard-drive.

Step three - Render in and save the image with a name like Color. The image won't have any shading so it may look a little strange.

Step four - Assign a new surface shader to the model. Once again, click the checkerboard beside the color attribute, only this time, in the create render node window, go to the Mental Ray section and create a Mib_amb_occlusion node. Now render the image and you'll be able to see the occlusion as the color. Now, you may need to adjust the occlusion settings.

Now we're going to take these two images and layer them on top of each other using a multiply filter. We'll be using Adobe Photoshop. The procedure will be basically the same for other image editing software.

Step one - Open Adobe Photoshop.

Step two - Open both images. Drag the background layer of the occlusion image and drop it on the color image.

Step three - Select the occlusion layer (this will be named layer1) and go to layer>layer style>blending options... The layer style window should open. Set the blend mode to multiply and now you've got a shaded fish using occlusion.