Masking hair - refine edge function?

[Herbert123]

Quick question: in Photoshop CS5/6 any mask/layer mask/selection can be quickly and efficiently refined with the "refine edge" command.

This simple tool, together with the matting tools in the layer menu seem to be the only two things missing in Photoline that cause a lot of headaches for me: for example, download the first chapter files from http://kelbytraining.com/books/compositing

Try to composite the blonde girl in the condo background.

To do a quality complex cutout with hair takes a minute at most in Photoshop now (check out the attached image).

No matter what I try, I cannot get the same quality masking done in Photoline, let alone that quickly and efficiently - the mask tools seem too limited, and it feels like I am working in Photoshop CS4 and before in this regard, taking a lot of manual labour and time to achieve reasonable results.

Unless I am missing something crucial? Maybe I am not using the correct workflow in Photoline? What is the best automatic method to mask hair properly in Photoline?

Any help much appreciated.

[Herbert123]

Oops, Entschuldigung: I meant this to be posted in the English section. My bad.

[Hoogo]

For the rough selection I've chosen the "Automask"-tool, even if it only creates hard edges. I held down shift and then made quite some selections of the background with little tolerance. Ignore the tolerance-slider, you can change the tolerance during selection when you move the mouse while holding the button.

This was fine for most of the picture, but left too much background within the hair.

Invert the mask (Alt+Shift+I), press Ctrl and remove more of the gray within the hair. This will create a mask with lots of wild pixels, but that's fine. Finally my mask looked like the attached picture.

There are also lots of spots at her forearms, her eyes, within her hair, the skirt... I've corrected that with the ordinary lasso-tool and Shift. Shift has a double function here, release the key when you started dragging.

Rough mask is done, now the refining.

There's a "enhance edge"-Button in the tool settings of all the selection tools. Size of 3 looked fine to me. Just press the button, wait, the program is working, but not showing it. After a while the mask is refined. Ignore that it looks too rounded at the hair.

Now switch to the border matting tool. It's hidden behind the automask-tool.
Select "edit soft edge" and press Apply. Wait.
Finally press "Extract selection". Select "Remove color border" and a white background. Play with threshold and radius, for me medium and radius 5 looked good.

Extract to clipboard, press ctrl+v in the new picture.
-----------------------------------
I'm not sure if I would approach this picture just like the PS-tutorial. I would treat her hair and the rest with different approaches. Hair in front of a nearly medium gray looks very good when you just place the layer without any masking on top of the new background and select "Overlay" as layer mode.

Here the hair is brighter than the background. A duplicate of the layer, turned to grey and treated with histogram correction and curves can create an excellent mask of the hair.

Sorry, I'm in a hurry ow.

[greenmorpher]

I vote with Herbert on this. I can’t imagine how complex it would be to program, but it would be great tool to have. I often have to take a portrait in situ regardless of the background and then extract the head and put it against another background. With Papua New Guinean hair I can usually get away with making a rough border with the lasso (either ordinary or that sticky one) then going around the outside with the eraser set to total softness because the hair is compact curls, but there are situations even with this hair, e.g. when the old background is much lighter than the new background, when it would be preferable to have the kind of facility Herbert talks about.

Best regards
Geoff

Geoffrey Heard
General Manager
Bushtrack Enterprises Ltd
http://www.bushtrackpng.com

[OldRadioGuy]

Quick question: in Photoshop CS5/6 any mask/layer mask/selection can be quickly and efficiently refined with the "refine edge" command.

This simple tool, together with the matting tools in the layer menu seem to be the only two things missing in Photoline that cause a lot of headaches for me ...

To do a quality complex cutout with hair takes a minute at most in Photoshop now ...

No matter what I try, I cannot get the same quality masking done in Photoline, let alone that quickly and efficiently - the mask tools seem too limited, ...

Maybe I am not using the correct workflow in Photoline? What is the best automatic method to mask hair properly in Photoline?

Herbert, I agree with you. PhotoLine's Quick Selection Tool (not to be confused with the Quick Selection Tool in Photoshop Elements, which is more like PL's Automask) is among the best for doing cutouts of objects with definitive edges, but when you have semi-transparent edges like hair, the faster way for me is to run a good external cutout plugin within PhotoLine. The application of the Border Matting tool to do hair just doesn't work for me. Perhaps something has been lost in the explanation translation to English. But I have tried numerous times, usually giving up in frustration and going to a plugin.

PS Elements 11 added "refine edge," and it is a significant time saver. I, too, would like to see PhotoLine add something like this smart "refine edge" function in addition to refining Automask to behave more like PSE's smart Quick Selection Tool.

Very seldom do I use PSE, however, because PL is much better in so many other ways.

Bob

[Herbert123]

Sorry Hoogo, I appreciate your effort, but that is not even close to what can be achieved in Photoshop with the refine edge command in just 30 seconds - and I am not mentioning the convoluted amount of steps you had to go through. Your approach reminds me of how it used to be done in earlier version of PH.

No, PH's refine edge command is absolutely wonderful for these type of complex masking jobs. I am unsure about the algorithms being run in the background, but they are good ones.

[Hoogo]

Well, the PS-tutorial doesn't show how the first rough mask was created. Though I've heard that the automask of PS does a good job here, I won't believe that it doesn't have the same problems with wrong selections or inclusions until I can see it. And the bigger part of my post was about this selection. How did this first rough selection looked like in the PS-Version?

From your screenshot I can only tell of "remove black matte" and "refine edge" (and are these steps really done in this order?). These 2 correspond to "Extract > Remove color border" and "enhance edge/border matting". Beside the border matting for semi-transparent areas, there's not that much difference compared to the PS-tools.

Then I'd like to have a closer look at these areas in the 100%-view of the model, not the version that was scaled down to 20% to match th background. Preferably with black background and white background. White background will tell how good the background was removed, black background will tell how good fine hair was preserved. It's hard to tell from that small final picture you've posted. And I spot a strange difference in your and my version: Your version shows a darker cyan and lighter hair at the top of her head! I think there was done something more to make the picture look good. Maybe her hair was bleached, maybe accidentally by the "remove black matte"?

Finally, there's someone selling training for PS, supported by Adobe. This doesn't mean that the whole thing is faked, surely not! But it means that you see the examples where the tools worked best, and not the examples where the tools created crap.

[bkh]

I just had a quick try at this — it's definitely good to know what other programs can achieve here. I chose a bit of hair:

before.jpg

and drew a mask using PL's border matting tool. (It's a bit strange that you have to "erase" the part which you want border matting to work on.)

matting tool.jpg

Then I used "Extract Selection" with a radius of 9 and high threshold and checked "remove coloured edge" and placed the result on a blue background. Here's the result.

after.jpg

I think it's not bad for an automatic tool, but of course I'd be curious to know how other programs/plugins fare.

Cheers

Burkhard.

[Hoogo]

Just stumbled upon the .psd-file with the boxer in that folder. That mask in there is surely NOT a good mask!