Tutorial:Masking in Photoshop
From Anipedia
Masking in Photoshop is a technique used control the opacity maps of layers. By doing this, region coloring is made simpler. Masks can be edited at any time, and can be linked or unlinked to a layer's transforms at any time. They can also be flattened or baked onto the layer. When you merge two layers, any masks will be baked and removed.
Masks are ideal for lineartists who wish to control the areas they color in so they can let their brushes move about a layer without worrying about overspill.
This tutorial will teach you how to create masks in Photoshop with the following steps:
- Creating a quick example image to work with.
- Adding a mask on that layer and explaining how it functions.
- Demonstrating how the mask works with the layer.
[edit] Step 1: Selecting an image to work with
In a typical situation, artists who work a lot with line like to select out areas to color in so they don't color 'over the lines'. Masking is a very effective way to achieve this. So, my example image to work with will be a circle shape outline. In this example, we have created a new layer in the layer window (Window > Layers, then click Create New Layer 
), and drawn the outline of a circle in it.
Here is what has been done so far:
The top layer has a circle outline, and the bottom layer is the default white background. Notice these are separate layers.
[edit] Step 2: Adding a mask to a layer
Adding a mask to a layer is the most simple part of the process. To do this, select the layer you want the mask to be on, then click the Add Layer Mask (
) button in the Layers window.
Your layers window should look something like this now:
Notice that the layer with the circle outline now has a chain link icon (
) next to it with another preview of what appears to be a white canvas. That 2nd image is your mask.
White on a mask means that area the mask corresponds to with the layer will be 100% opaque, meaning when you first add a mask, your layer is unchanged. The link icon () means that if you resize, move or transform the layer or the mask, the other one will go with it. If you click the link, it will go away, and the mask and the layer can move separately.
[edit] Step 3: Ways to manipulate your mask
A mask has a simple function - complete control over the transparency of your layer. A mask is monochrome, meaning the only values it reads are from white to black. White means opaque, black means transparent. Try this. Select the mask in your layer (you must click on the preview of the mask in the layer), select the Paint Bucket in your toolbar, and fill the mask with black. Your circle should disappear. If you fill it with white, it will reappear.
Now let's apply this in a practical situation. Let's say that you want to only have your colors appear inside the circle. Therefore, you will want to select the area you want to color in and apply it to the mask. There are several ways to do this, but this tutorial will show you the two most common:
- Selecting and filling an area using the Quick Mask Mode. Quick Mask Mode can be activated by hitting Q or selecting the Quick Mask Mode button (
) on your toolbar. Now select your circle outline layer. We want to only color inside of the circle, so we want to mask out everything around it. With Quick Mask Mode, the brush now turns into a selection tool. Whatever you color will turn a semi-transparent red. This is not part of the layer, it's actually part of a selection. With Quick Mask Mode on, use your brush to sweep across your layer. Then turn it off. Whatever was red is now part of a selection.
You can use this selection to then fill it with either white or black in your mask. If you accidentally selected the opposite part of what you wanted to mask, you can invert selection by hitting Shift+Ctrl+I, or going to Select > Inverse Selection.
- Using the selection tool. In this case - since we're dealing with a simple circle - putting the selection tools to our disposal would be a better option. The Magic Wand
and Quick Selection 
(CS3 and up) tools are great for separating linework from white space. You can find them in your toolbar and you can toggle them by hitting W or holding your mouse down on either one to reveal the other to select it.
In this case we'll use the Magic Wand tool. Click the layer with the outlines, and select the outside space of the circle. The Magic Wand will create a selection of the transparent area outside the circle. Now select your mask, use the Paint Bucket tool and fill it with black.
Now deselect your selection using Ctrl+D or going to Select > Deselect. Now take a brush and scribble inside the circle layer. Make sure you don't have the mask selected anymore or you will paint inside the mask and not the layer. You should notice that as you color outside the lines, your brushstrokes do not appear where you filled the mask with black.
If you put a 50% gray on the mask, color on the layer in that area will appear halfway transparent. The last two concepts are illustrated in the picture below.
If you don't like the edges that selection tools made for you in the mask, you can always fine-tune them by either changing the tolerance level on the Quick Selection tool (located in the tool options below the file menus), or use your brush to manually change the edges in the mask itself. Don't forget you can also use the Quick Mask tool to make new selections.
Masks can also easily be removed by selecting the mask and hitting the Delete Layer button in the Layers menu. It will ask you if you ant to apply the mask to the layer before removing. If you want it to keep the mask work you did, select Apply. If you want everything to go back to the way it was before you started masking, hit Delete.
