Exp: Selections and Masks

[photoken]

Looking forward to your post regarding a possible chisel-like mask refinement technique in PL.

OK, it's really simple:

• I used Herbert123's technique of creating a Minimize adjustment layer (as a child layer) for the original mask.
• Create a new mask (as a child layer) for that adjustment layer.
• With that new mask as the active layer, use Tool...Color...Invert (Ctrl+I) to make this mask solid black. That means the Minimize adjustment layer will not be applied anywhere to the original mask.
• Paint with white on the adjustment layer's mask wherever you want the original mask to move in (by whatever amount you set in the adjustment layer's settings).

The layers look like this:

blue masked layers.png

The result:

blue masked result.png

The edge of the original mask only moves in where the white has been painted. Brush size doesn't matter, and you can paint as sloppily as you want -- as long as the white area straddles where the edge of the original mask is, you're golden.

Of course, this doesn't even come close to being as convenient as having a Chisel tool, but if I correctly understood what you described, the result should be the same. I guess you could always record an action to create the Minimize adjustment layer and its mask and invert its colour, and then assign a hotkey to that action....

[Martin Huber]

Of course, this doesn't even come close to being as convenient as having a Chisel tool,

If I understand that correctly, the Chisel tool is just a filter brush using the Minimize filter. So you can do just that in PhotoLine: Use the Filter Brush with the Minimum filter and a filter size of 1.
If you edit the selection, you have to enter the Edit Mask mode first.

Martin

[photoken]

I I understand that correctly, the Chisel tool is just a filter brush using the Minimize filter. So you can do just that in PhotoLine: Use the Filter Brush with the Minimum filter and a filter size of 1.
If you edit the selection, you have to enter the Edit Mask mode first.

Wow! That looks like it, alright. Thanks!

[Herbert123]

Of course, this doesn't even come close to being as convenient as having a Chisel tool,

I I understand that correctly, the Chisel tool is just a filter brush using the Minimize filter. So you can do just that in PhotoLine: Use the Filter Brush with the Minimum filter and a filter size of 1.
If you edit the selection, you have to enter the Edit Mask mode first.

Martin

Martin, I wish we could use blend modes when painting. A common trick to remove and clean up grey-ish transparent areas after mask extraction is to use the painting brush in overlay mode when editing the layer mask. I do not believe that is possible in Photoline?

[Martin Huber]

Martin, I wish we could use blend modes when painting. A common trick to remove and clean up grey-ish transparent areas after mask extraction is to use the painting brush in overlay mode when editing the layer mask. I do not believe that is possible in Photoline?

Yes, that is currently not possible.

Martin

[Juan]

I wish we could use blend modes when painting

Please, I add my vote to this.

Cheers,
Juan

[Herbert123]

At this point I feel it would be useful to work on a perfect photo with a prepared background for extraction in Photoline, and see if we can create a high quality mask. I have played around with the border matting tool, and I think I figured out the basic workflow.

I compared the quality with Photoshop's refine selection tool, and the results are quite similar. Photoshop's area selection tool seems a bit more intelligent, though, and it seems somewhat easier to arrive at an acceptable result. The workflow is actually remarkably similar, come to think of it. In Photoshop there is no need to define the background area, though - only the subject needs to be selected before working on the hair matte.

I made a quick tutorial movie - warning: this is a very specific example, and the mask is good enough for this particular background. But it is merely a quick test to see how Photoline's extraction tools work with high-quality prepared photos. Ideally the extraction tools should be able to extract a high quality mask under these circumstances. That does not seem to be the case quite yet. I feel the tools could be improved a bit more - especially the colour contamination removal: it did not work that well for me in this case.

Here is the tutorial movie (you may be redirected to another page - click on the "qui" link):
http://estructor.altervista.org/pl/masking1.mp4

I do make use of a trick to remove the grey ghosting (inner glow layer effect). Another method is painting in the mask with the blend mode set to overlay, but PL does not support that, unfortunately. Another missing feature is an option to render each layer effect to its own layer. That would allow me to add a mask to the layer effect, and remove it from the jacket edges, if I wanted.

Here are the test files:
http://estructor.altervista.org/pl/1.jpg
http://estructor.altervista.org/pl/2.jpg
http://estructor.altervista.org/pl/3.jpg

Let's try to come up with a near perfect selection for the two girls! This should be possible with Photoline. My results so far:

test1.jpg

test2.jpg

[Hoogo]

Martin, I wish we could use blend modes when painting. A common trick to remove and clean up grey-ish transparent areas after mask extraction is to use the painting brush in overlay mode when editing the layer mask. I do not believe that is possible in Photoline?

You could group the layer mask, add another layer and turn the layer mode to overlay. But I think that a curves adjustment layer added to the mask will serve the purpose even better.

[TT1]

@Herbert123

Thanks for taking the time to make the Masking Tutorial video at...
http://estructor.altervista.org/pl/masking1.mp4
I find it very useful, and only wish there were more tutorials like it.

Also, I've been directing my contacts to the excellent review at...
http://stargazerslounge.com/topic/16009 ... try1684199
Assuming you're the same Herbert123... It's a well written review. I wish there was better coverage of PhotoLine. Imagine how PL would fare if there was better focus on marketing it. However, I'm glad the 'brothers' are continuing their diligent work on the program.

[Alan9940]

@Herbert123

First, thank you for the video tutorial. As someone else already mentioned, I wish there were more tutorials like this on PL. But... There's always a 'but' isn't there?

I followed along with the tutorial, but my version (18.52) of PL doesn't have a destination of 'mask' in the Extract Selection. Therefore, I could only follow the tutorial to a point and I'm not sure if the artifacts in my result are due to this fact. The BM tool seems like it would be tailor made to work with things like fly-away hair, but I've yet to get the hang of it! Mostly what I see is blurry hair areas where, and I'm totally guessing here, the compute area may have overlapped and/or come in contact with the keep area? The very fine strands of fly-away hair seem fine; my issue is around the immediate head area (see Extracted attachment.) If you look closely at the solid hair area right next to the head, it looks blurry vs the original. And, what's with the artifacts in the hair to the left of this area?

Just for fun, I worked up a mask in PS using the Refine Edge dialog (see Testpic1_PS) and, again, another mask created using Topaz Remask 4 (see Testpic1_RM.) Both of these pics reveal the gray background in hair areas where neither tool was able to compute successfully on its own, but these small areas could be further refined manually on the mask via a normal brush. And, in both cases the time required to obtain a pretty decent mask was much less than the effort required in PL. Personally, I don't care how long these tasks take--as long as I can get a really good mask--but, others might. All that said, after doing these exercises I'm really starting to form a new appreciation for Topaz Remask 4! For me, I just can't obtain the same level of mask quality using PL tools...I wish I could, but I can't.

Any input here that might help me understand what I'm doing wrong in PL and why I'm seeing the result that I am would be greatly appreciated. Thank you.

Best regards,
AlanH

P.S. Hey Ken, I tried the "chisel" idea using Minimum via the Filter Brush (setting of 1) around the Jen's arm and lower torso area to remove that bit of orange glow--and the tool did remove it--but I also got a darkening (almost like a dark halo, if there could even be such a thing) around both sides of the edge. It's like there was some kind of built-in brush softening going on. Dunno. Have you tried this? Were you satisfied with the result?

[photoken]

P.S. Hey Ken, I tried the "chisel" idea using Minimum via the Filter Brush (setting of 1) around the Jen's arm and lower torso area to remove that bit of orange glow--and the tool did remove it--but I also got a darkening (almost like a dark halo, if there could even be such a thing) around both sides of the edge. It's like there was some kind of built-in brush softening going on. Dunno. Have you tried this? Were you satisfied with the result?

Hi Alan,
Hard to guess what you're seeing -- can you post a screenshot(s) at maybe 400% of the area you're working on? Showing your mask and the result after using Minimum?

I haven't masked her torso and purse for this Experiment, and I probably won't. When creating the "isolation" mask the first time, I saw all the problems with the edges in the image (I listed some of them in the first message here). So I decided then that the only way to get an absolutely perfect mask around her torso would mean hand painting. I used the Free Lasso tool to make an initial selection and created a mask from that. Then I zoomed in to 400% and painted with a brush, (varying the size from 2px to 7px as necessary) along every pixel of the edge. I also varied the opacity of the brush to get an appropriately soft edge to the mask where necessary. There were also places along her legs where there was some kind of "sensor spillover" which created an abrupt "jog" in the edge. I had to paint in further into her leg before and after those "jogs" in order to get a smooth line to her leg. All-in-all it was very tedious, for sure, and involved making constant decisions and compromises about what should be in or out.

Here's what I mean:
At 400%, you can see 5 areas

Jen edge.png

Area 1 is the fabric.
Area 2 is a narrow band where the fabric starts curving away from the camera. It's slightly darker in this section, but further up along her sweater it's got a light blue colour. These effects are probably due to the curved fabric reflecting (or absorbing) the ambient light back to the camera.
Area 3 is a reddish band of colour that is like a blend of the fabric colour and the background. This band looks like it's the blurred edge of the fabric.
Area 4 is a light band of colour that looks like it's some kind of sensor or in-camera processing artifact.
Area 5 is the background.

You can see the difficulties that any automatic selection method is going to have with those areas -- what should be considered "in" and what is "out"? I suspect that an automatic selection is going to go back and forth between considering the edge between Area 3 and Area 4 and the edge between Area 2 and Area 3 as being the "real" edge.

O course, there's always the option of using a less extreme method of fine-tuning the mask, and I'm certainly interested in seeing the results from other techniques.

[Alan9940]

Hello Ken,

I gotta tell ya dude, your posts always seem to lead me in some kind of direction! I may not follow exactly what you're doing, but the seeds are planted and it gets me to thinking!! That's not so easy nowadays for this old dude!!

Anyway, based on your last post and reference back to others I did another little trial this afternoon... Using the Automatic Lasso (which I've never really used because I couldn't get a useful technique working until I read a recent thread that, I believe, Herbert123 responded to, but don't quote me on that), I made an initial small selection along Jen's back torso area trying to get about the same piece you were discussing in your last post. Btw, even with the "fuzziness" on this edge the AL worked really well to follow that line! I plan to start using this tool more...

Continuing on... Once I had this basic selection, I created a mask. Then, I created two child Minimum adjustment layers with size of "1" leaving one child layer on full bore and hiding the other child layer with a black mask. Then, with a brush I painted back only those areas along the line where I wanted an additional "chunk" taken off. This really cleaned up small remaining green/teal and yellow areas along that line! Not bad...not bad at all (see attached.)

Regarding your comments about what we're looking at along that line, IMO it could be some JPEG compression artifacts, it most definitely is reflected yellow light off the cab, and looking at that line at 400% it reveals a blur radius of about 5 - 6 indicating a very low quality image. Even given JPEG compression, etc, I doubt this picture was taken with a very high quality camera.

Thanks, again, for getting the 'ole synapses firing!!

Best regards,
AlanH

[photoken]

Btw, even with the "fuzziness" on this edge the AL worked really well to follow that line! I plan to start using this tool more...

That's good to know. Thanks! I'll have to start using AL some more, too -- if it proves better than I expected, then I've learned something valuable.

Continuing on... Once I had this basic selection, I created a mask. Then, I created two child Minimum adjustment layers with size of "1" leaving one child layer on full bore and hiding the other child layer with a black mask. Then, with a brush I painted back only those areas along the line where I wanted an additional "chunk" taken off. This really cleaned up small remaining green/teal and yellow areas along that line! Not bad...not bad at all (see attached.)

Not bad at all. I would never have thought of combining two Minimum layers like that!

Regarding your comments about what we're looking at along that line, IMO it could be some JPEG compression artifacts, it most definitely is reflected yellow light off the cab, and looking at that line at 400% it reveals a blur radius of about 5 - 6 indicating a very low quality image. Even given JPEG compression, etc, I doubt this picture was taken with a very high quality camera.

I completely agree -- it might have been a high quality camera, but the shot looks like an obvious hand-held "grab" shot of action while the filming was going on. No telling how much quality was lost due to camera shake, and subject movement, etc.

On somewhat of a side note, your image shows a weird thing that I noticed when I first saw the original: WTF is going on with the bottom edge of her forearm? If I was doing the isolation of her for publication or something, I think I'd have to use the Copy brush to clone some of her skirt over the bottom edge of her forearm to smooth out the line of the lower edge as it crosses the fabric onto the background....

I gotta tell ya dude, your posts always seem to lead me in some kind of direction! I may not follow exactly what you're doing, but the seeds are planted and it gets me to thinking!! That's not so easy nowadays for this old dude!!

Mr Natural 01.png

[Alan9940]

On somewhat of a side note, your image shows a weird thing that I noticed when I first saw the original: WTF is going on with the bottom edge of her forearm? If I was doing the isolation of her for publication or something, I think I'd have to use the Copy brush to clone some of her skirt over the bottom edge of her forearm to smooth out the line of the lower edge as it crosses the fabric onto the background....

Yeah, leave it to you to nitpick!! Heck, I dunno...I didn't pay attention to that line at all. Forget the forearm is even there!!

Best regards,
AlanH

[photoken]

Forget the forearm is even there!!

Easy to do when we're talking about The Jen!