Wednesday, June 10, 2015

Layer Masks and Modes Tutorial

After seeing many questions regarding how I create some of the edits that I do, I decided to do a tutorial on layer masks and modes.  For this tutorial, I'll be using PSE 12 instead of GIMP since it's a fairly translatable technique (and also PSE 12 came free with my Wacom tablet ;p ).  I'll also provide explanations for Gimp users where menu selections are a tad different.

Before I begin, this is my photo after basic retouching edits (frequency separation, dodge and burn, heal tool, liquify, and sharpening).  So I've cleaned up blemishes, evened out shadows, fixed distortions, etc.  Now I'm going to enhance shadows and highlights with layer modes using masks.



The very first thing you will want to do is duplicate your layer.  If using PSE or Gimp, there are two options to this.

  • right click and select "duplicate layer" from the menu 
  • go directly to the Layer menu at the top of the screen and select "duplicate layer".



PSE 12 users will have a pop up dialogue box asking you to rename the layer.  This is optional and I only recommend renaming if you're going to be working with more than 3 layers as it can become confusing which is which. Gimp users, you won't receive any pop up box.


After clicking OK, you are going to play with Layer Modes next.  You'll notice in the layer box you have a drop down menu that defaults to "Normal".  This is the same in both PSE 12 and Gimp.  

 

If you click on this menu, you'll see a drop down menu with 25 different layer mode options.  Gimp users will only have 21 options.  For this tutorial, I will be using Multiply and Screen.


Once you've selected "multiply", your image should look darker.  If it doesn't, make sure you have the top layer selected and not the bottom layer (aka the background layer).



Here's a side by side of the background layer (layer 1) and the multiply layer (layer 2).


Obviously, I don't want my entire image darkened like this and only want that burned look to appear in certain areas.  This is where using your layer masks comes in handy.  Here's a little explanation of how masks function:
In basic layer masking, you use either a white or a black layer mask.  White layer masks will reveal the layer on which you apply it.  Black layer masks will conceal the layer on which you apply it.  So, if you are working with two layers (background and duplicate) and you add a white layer mask, you will only see the work on the duplicate.  You will not see anything from the background layer (initially).  If you add a black layer mask, you will no longer see the duplicate layer and will only see the background layer.
Choosing which color of mask is important as it can save you time in guessing what you're changing.  For me, if I want to keep the majority of the changes on my duplicated layer (layer 2), I use a white mask so I can see more.  If I want to only use a few parts of my duplicated layer, I use a black mask so I only see the background layer (layer 1) when I begin.  
To hide or show select areas of your duplicated layer, you will be using the paint brush tool to paint directly on the layer.  When you paint, it will be opposite.  If you have a white layer mask and wish to hide what is showing on the duplicate layer (layer 2), you will use the color black.  If you have a black layer mask and wish to show what is concealed on the duplicate layer, you will use the color white.  
Layer masks allow you to erase or paint on your image, without permanently altering it.  If you make a mistake, you simply switch your paint brush color back to the main color of your mask to "erase" your mistake.  This is thousand times better than using the actual eraser tool and as you experiment with this, you can use selection tools and the paint bucket to speed up the process too.  
So here's some screen shot demos and step by step for adding your layer masks.

First, PSE users, you will click on the layer mask icon.  This defaults to a white layer mask.  You can also add a layer mask by going to the top menu, selecting "Layer"  - "Layer Mask" -  "Reveal All" (for a white mask) or "Hide All" (for a black mask).  Gimp users, life is easier for you.  Just right click on your duplicated layer and select "Layer Mask" from the menu.  You will receive a pop up box that defaults to White, but can also select Black if you wish.  Just click the "add" button to add it.




Using the icon method to add a layer mask, you should see a little white square next to your duplicated layer.  Make sure when you are working with layer masks that you have the layer mask itself selected before you paint (it will show the turquoise border around it as in the screenshot below).  If you don't have it selected, you will be painting on the actual layer which isn't reversible if you make too many edits--the undo button has a limited history!
Gimp users, your box will be bordered in white, which doesn't stand out at all.  So make sure that it doesn't show with a black border around it because that means you have a different layer selected.





Now it's time to paint on the layer mask.  First select your paint brush tool.  PSE users, a menu will appear at the bottom of your screen to adjust the brush's size and opacity.  For this, leave the opacity at 100%, but change the size as you need.  When I'm increasing shadows or highlights with this method, I typically use a basic round fuzzy brush.  However, feel free to use whatever brushes you'd like to experiment.  Gimp users, to change brushes just double click the paintbrush icon in your tool bar and a dialogue box will open where you can select style, size, and opacity.




Brush selection menu -- I have a lot of options from adding creative commons ones to my PSE.
 Next I'm going to change my brush color as I need a black paintbrush to paint on a white layer mask.  This is how I'll hide parts of the duplicate layer (layer 2), letting the background layer (layer 1) show through.  To do this, click on the switch arrow so your foreground color changes from white to black.



To begin painting just place your cursor on the main image display and start painting away!  You'll see your layer mask thumbnail begin to change and show the areas you've colored in black as you go.  If you don't notice any change--double check that your layer mask is what is actually selected (PSE = turquoise and Gimp = white -- if not selected it will have the black border).



Since the thumbnail is tiny and hard to see details, you can check if you missed a spot by changing your view to only the layer mask.  Do this by holding down the ALT key on your keyboard and left clicking on the layer mask thumbnail.  To return to normal viewing, repeat this step.  Gimp users can do the same step.


Some small tips and tricks!  You can run the Gaussian blur filter on your layer mask to smooth out the edges of your paint brush strokes.  You can also change the opacity of your brush itself if you want to reveal/conceal less on the layer mask.  

Another technique you can apply is changing the layer opacity levels to alter how much or little the layer mask will reveal/conceal of your duplicated layer.  Just click on the drop down for the opacity and move the slider as desired.  Gimp users, you will use the opacity slider at the top of the layers dialogue box.



When you have your duplicate layer with mask adjusted to how you like, you can save your PSD file with the layers intact.  This will allow you to reopen the file and make changes without having to start all over again (for example, maybe you print the image and it's too dark, so you can reopen the PSD file and lower the opacity of the layer mask to lighten the image).  This is how I store my files in case I need to make later adjustments.  If you want a jpeg image to display, after you've saved your file as a PSD file, do "save as" and select jpeg to create a jpeg copy of the file that can be printed.  Gimp users, you will want to export your file as PSD (you can also do save as XCF -- which is a file that can only be opened in Gimp, but I recommend PSD as it saves and opens faster than XCF files).  Then you can export your file as a jpeg to have your printable image.

This is my final image after doing both a multiply layer with a mask and then a screen layer with a mask.  Below is a screen shot of my layers with masks, the final image, and side by side comparisons of the various layers.

Layers with Layer Masks

Final Image after masking edits

Unmasked Layers (Original/Background, Multiply, Screen)
Unmasked Multiply Layer vs.
Masked with Painted Areas Multiply Layer

Unmasked Screen Layer vs.
Masked with Painted Areas Screen Layer
Original image compared to final image with all layers visible.  It's a subtle change as I didn't want super drastic for this image.
Also, wanted to give special thanks to Chanelle Culler for being my lovely model here!

1 comment:

  1. Good work. Like how you show the process both for GIMP and Photo$hop.

    ReplyDelete