Masks : Essential tools for your CG workflow 1/2

update 1-oct-2009/ thanks to Peter, I’ve learned that Multimatte workd by default with object IDs. The topic has then be corrected/completed

update 27-oct-2009/ added some info on the “Affect channel : All channels” option of a material

When it come to archviz rendering, one of the first thing to learn is to create good masks to retouch materials or objects individually in photoshop. If the most popular alpha mask allows you to cut out the entire rendering from his background, other type of masks (based on elements or objects) became essential for an efficient workflow in archviz.

I remember times where I did a second rendering with the same point of view, where every material was replaced by a self-illumated conterpart in order to save one or several DIY “masking pass” …a time consuming method, either on material editing or rendering. I even remember cutting out glass masks for buildings directly in Photoshop. Brrrrr :)

Hopefully, render engines have implemented numerous ways to extract these multiple layers during the rendering of the principal image. Those render elements may then be composed afterwards, leaving the artist a greater control on the final result. I’ll make a simple example with my usual pipeline : 3dsmax – Vray – Photoshop, but this can be applied to any 3d program that gives the opportunity to save differents passes of a unique rendering.

We’ll take as example a rendering where glass has to be enhanced, like in 95% of architecture renderings. (Could you make the glass more realistic please ? ;) )

Here’s quite complex facade with glass, displaced perforated metal in front of it, furniture etc. Out of the box render without any post production looks like this :

To improve the glass, we’ll need 3 more passes : One for the reflection, one for the refraction (the model behind the glass) and a good mask to affect only the glass in photoshop.

For this exmaple, we’ll try different render elements and see what are the pros & cons of each. Beware that the more elements you want to be included in your render, the more RAM it will need for the EGB buffer. (Also, I’ve red that a big number of passes will slow down the overall rendering time a bit)

1) VRayWireColor

My favorite. The biggest advantage is that it requires almost no setup. I use it as a backup masking pass, in addition to MultiMatteElement. The only thing to remember it that it uses the color of wire objects, so you have to give differents color to your object in order to avoid confusion.

fig 4a. Wire color render pass

Pros

• Nothing to setup
• Distinct mask for every object
• Works with opacity maks, transparency if needed.
• Can be previewed in the viewport with “diplay object color” instead of material color
• Everything in a single RGB pass

Cons

• You objects need to be more or les attached by material (frames & glass detached, this is sometimes a big constraint)
• You need to give those object different colors in order to work
• No “pixel perfect 255 levels gradient” precision, more something like 50-100 pixels precisions if your object colors are well picked

fig 4b. Choose Object color for shading to preview the pass

fig 4c. the viewport preview of the pass. No transparency, indeed...

2) MultimatteElement

The professional one. This works by default with Object IDs but you can use it with Materail IDs too (just check the box in the rollout menu) You set it up by giving a Material ID to the material from which you want to receive a mask. A RGB is then generated at render time, one material ID by color channel. Be sure to give different names in the RenderElement setup screen to avoid overwriting of Multimattes on top of themselves.

fig 5a. Give you mat IDs here

fig 5b. and you'll receive this ...

fig 5c. ... and this... and more if needed

Another great tool for masking transparent objects is to maks them “transparently” with the “Affect channels : All”

Note that this technique usually mess up with all other channels like GI pass, Wirecolor etc. That’s a big drawback and maybe a future update would allow us to choose to affect alpha & Multimatte only…

fig 5d. The RGB pass and his mask, notice how the mask overlap each other, particularly usedful to cut out the glass railing and the glass behind separately

This option is locatated in the refractive options of a VRayMaterial

fig 5e. Choose "All Channels" in the drop down menu

Pros

• The professional choice :)
• Pixel perfect mask, 255 levels of precision.
• Works with opacity maps, transparencies, etc.
• Object or Material based
• Works perfectly with “All channels” chosen in the refractive options of a material

Cons

• A tad long to setup if you have numerous materials in your scene.
• A new pass every 3 masks (increase RAM usage)
• Limited to 15 materials ID if used with material IDs (3dsMax limitation)
• No real way to preview it

3) Zdepth

Zdepth is not really used in first place for masking, but we’ll see later how it can save time in some particular situations !

fig 6. There is always something beautiful about Zdepth :)

Pros

• Useful in very specific masking situations

Cons

• Useless most of time (for masking purposes)

4) VRayObjectID

I initially thought that this pass might be useful, but I must have been mistaken… It’s useless in this case!

fig 7. Object ID pass

Pros

• None

Cons

• Everything (No AA, not customizable, etc. )

(If someone could tell me the utility of this pass, I’m intrested :)

5) VRayRefractionFilter

The joker. Not really a good workflow, but it made my day once so it deserves to be here ! A VRayRefraction pass could do the same, when all his non-black pixels are made white, but then you’ll lose AA.

fig 8. Refraction ID will show only the refractive materials.

Pros

• No setup, just include it.

Cons

• Only useful for glass in archviz
• Doesn’t work when you have multiple refractive elements, will blend all the masks together

That’s all for today, in the second part of this tutorial, we’ll see how to use the above masks to improve our rendering locally !