Grain Merge Layer Blend Mode in PL: Wavelet Decomposition

[cathodeRay]

EDIT (5th Feb 2017): I have edited the title of this thread to reflect the fact it now includes, in post 15, a 'recipe' for doing wavelet decomposition in PL.

As a step on the way to getting wavelet decomposition in PL, I have been experimenting with ways to emulate what GIMP calls Grain Merge layer (blend) mode. The maths are simple: C = A + B - 128, or possibly (this is relevant, as in the order of doing the maths - see below, in the way the stack is set up) C = (B - 128) + A, where A is the lower layer, B is the upper layer and C is the result. Even I - as I understand these things - do realise that mathematically these are the same, but it seems it has to be set up as the second way to be do-able in PL.

The GIMP manual page is here, and the images I have used are the ones on the left hand side (two original images near the top, and the Grain Merge example further down).

I appear to have achieved this by (note: all layers are set to 32bit - this seems to be important, so the out of 0-255 range numbers get handled properly - but see my question at the end...):

1. Place the lower layer
2. Add the upper layer
3. Add a new rgb layer filled with 128 grey (ie 128,128,128) above the upper layer
4. Set the grey layer's blend mode to Subtract
5. Group the grey layer and the upper layer and set the group to Draw Isolated

The Group is now (B - 128) and so far so good: the colour watcher panels show negative byte numbers for the group layer as expected (ie when the upper layer value was <128, as was the case at point 1).

Group layer.jpg

Now set the Group's layer blend mode to Linear Dodge (Add) - the is the '+ A' bit

And Bingo! The end result is visually indistinguishable from the GIMP manual's own example of Grain Merge!

GIMP Grain Merge.jpg

PL Grain Merge.jpg

This is what the layer stack looks like:

[Grr... see next post - can't add more than three attachments...]

However, there are some differences behind the scenes. These can be seen by

(a) setting the GIMP version to Difference blend against the PL version - some minor differences in the whiter parts
(b) checking the colour watcher panels rgb byte figures for the PL version: the actual byte values can be > 255
(c) turn on mark extreme values - the > 255 values get marked in the PL version

Colour watcher panels with some values > 255 at point 3:

[Grr... see next post - can't add more than three attachments...]

So what appears on the monitor, ie what I see on the screen is OK (the > 255 values are simply clipped to white), but behind the scenes the pixel values are out of range. Does this matter? I've a hunch it might, because PL works of the pixel values, not what we actually see. I did try to think of a way of mapping the > 255 values in the PL version to 255 - maybe some sort of adjustment layer? - but no real dice. I did sort of manage it by doing Merged Copy > Paste then set that copy to 16bit - both the numbers go back to 0-255 and the difference blend between the GIMP version and the PL version not shows no difference but the problem is that merged copy sets the adjustment, and of course the whole idea is to achieve this as a dynamic non-destructive technique...

Any thoughts would be very welcome!

cathodeRay

[cathodeRay]

The other two images:

The layer stack:

Layer stack.jpg

RGB values out of range (> 255) at point 3:

Out of range RGB values.jpg

cR

[bkh]

If you need to work in 8 or 16 bit mode (avoiding overflows/underflows), you can obtain the same result if you

• set the blend mode of the top layer to normal, 50% opacity
• add a histogram correction adjustment layer (or a curves adjustment layer) mapping 64 to 0 (black) and 191 to 255 (white) above the top layer (in the main layer stack, not as a child layer).

This may cause slight banding in 8 bit mode, but if you want to do wavelets, you'll probably want to use 16 bits anyway. (Creating a layer stack with full precision at 8 bits is doable, but quite complicated.)

Cheers

Burkhard.

[cathodeRay]

Burkhard - as you have probably come to realise by now, I like to avoid anything 'black box' or voodoo, and furthermore like to both be able to understand in pretty much lay terms what's happening and be able to verify that two plus two does indeed equal four.

I did think about how HG adjustment, curves or indeed a custom filter adjustment layer might do what is needed, but it seemed to me that both HG and curve adjustments are mapping/scaling adjustments, ie always at the end of the day a multiplication operation, the only exception I can see being the bias setting in the custom filter dialog, which does add or subtract a fixed number. A normal blend on the other hand is at heart an addition: in the case of 50% opacity it is C = (A + B)/2 or more generally C = (A x opacity A) + (B x (1 - opacity A), with the opacities expressed as percentages. So we can get the addition like that - as far as I can imagine (haven't tested it) normal blend at 50% scaled by a factor of two would be the same as Linear Dodge (Add) ie C = A + B.

Where I struggled and still do is how to do the subtraction without, err, using subtraction. When I tried your method (at 16 bits) (place background, add upper layer, set upper layer blend to normal at 50 %, add a HG adjustment layer to the upper layer mapping 64 in to 0 out and 195 in to 255 out), I get something similar to grain merge, but overall it's more more washed out. I just can't see how expanding to 64 -195 range to 0-255 amounts to minus 128 across the board???

The GIMP manual is a bit acerbic. It does say (near the top) that out of range numbers get clipped, unless otherwise stated. However, elsewhere, we have this:

"Grain merge and Grain extract are my favorite layer modes. They're versatile and useful for many purposes - they're powerful, yet simple. They can be used for texturing, lighting, shading, beveling, embossing... many things.

Grain merge can basically be understood as converting the top layer's pixel values (0 to 255) into signed integers (-128 to 127), then adding them to the pixel values of the image below. Grain extract is the opposite of that. It converts the top layer to signed integers, but then subtracts them from the image below."

So, using 32 bit to allow out of range numbers (and floats behind the scenes), my 128 grey layer in subtract blend mode does the 0 to 255 to -127 to 127 bit. The thing here seems to be that during the maths you do have to go out of range: if you clip, you add distortions, as in the starting point for the addition is artificially high (0, rather than minus whatever, so the final value (C) is too high.

Reviewing the above, it did occur to me that 'converting the top layer's pixel values (0 to 255) into signed integers (-128 to 127)' is in fact a mapping operation, albeit by subtraction rather than scaling... so I tried setting the HG inputs at 0 to 255 and the outputs at -127 and 127, but it wasn't having it, neither at 16 or 32 bits - the -127 gets 'reset' to 0...

cR

[bkh]

Where I struggled and still do is how to do the subtraction without, err, using subtraction. When I tried your method (at 16 bits) (place background, add upper layer, set upper layer blend to normal at 50 %, add a HG adjustment layer to the upper layer mapping 64 in to 0 out and 195 in to 255 out), I get something similar to grain merge, but overall it's more more washed out. I just can't see how expanding to 64 -195 range to 0-255 amounts to minus 128 across the board???

Sorry, I meant 191, not 195. That should explain the difference. Since you have 0.5 A + 0.5 B, you have to multiply by 2 and subtract 128 to get the desired result, i.e., you have to map a value x to 2x-128 to get the desired value. Now it's easy to check that the preimage of 0 is 64, and the preimage of 255 is 191.5, these are the values you have to use in the histogram panel to get the linear transformation. No voodo involved, just simple math. The only tricky bit is to avoid overflows at intermediate stages.

Cheers

Burkhard.

[cathodeRay]

Burkhard - I did wonder where the 64 and 195 came from (more voodoo? only coming from you that seemed most unlikely!) and then realised 64 was half of 128, so perhaps you had meant 191 (255-64) and tried that - in effect mapping the central 127 (128) values to 0 to 255 - but the end result still didn't equal GIMP's Grain Merge - still too washed out. Sort of makes sense, I thought, because aren't we losing data ie that in the cut off tails?

However, thanks for explaining the simple voodoo I mean maths, which I do now get: (2 x 64) - 128 = 0 so 64 goes to 0, and (2 x 191.5) - 128 = 255 so 191(.5) goes to 255 etc, but as set up (see earlier description) I still wasn't getting the correct result. It then occurred to me that as I was setting the HG as an adjustment layer on the upper layer, the HG was working only on that layer, not the blended layers, and sure enough if I do merge copy > paste > HG adjustment on the merged copy layer Bingo! I do get the right result. The only thing is that, as noted elsewhere, that merged copy step 'sets' the image at that point/layer, ie we have lost the ability to do dynamic editing; instead, to effect a change, it would mean deleting the merged copy and starting all over again.

In passing, and I've meant to ask this before, and it's obviously relevant here, is there any way to get an adjustment layer to work on the blended pixels (as in it's pixels as modified by any blend mode currently in operation)? I thought this was possible (even at one time thought it was the default) but it now seems an adjustment layer can only work on the pixels on the layer it is attached to. I also tried merge copy > paste as virtual layer, but it's greyed out, and grouping and draw isolated to get the group to 'send up' the final result, but no dice.

cR

[bkh]

It then occurred to me that as I was setting the HG as an adjustment layer on the upper layer, the HG was working only on that layer, not the blended layers, and sure enough if I do merge copy > paste > HG adjustment on the merged copy layer Bingo! I do get the right result. The only thing is that, as noted elsewhere, that merged copy step 'sets' the image at that point/layer, ie we have lost the ability to do dynamic editing; instead, to effect a change, it would mean deleting the merged copy and starting all over again.

The HG adjustment should work as expected if you just put it immediately on top of the 50% layer (in the normal layer stack), instead of using it as a child layer. I should have explained this better, and will fix my post above.

Cheers

Burkhard.

[cathodeRay]

I think I'm probably hitting a brick wall here (I've moved on to doing the actual wavelet decomposition, using a 'recipe' put together from various websites and the decoding of a GIMP script for wavelet decomposition - not for the faint-hearted, as there is virtually no documentation). The blur then grain extract (subtract then add 128) bits seem to work OK, probably because the layers are very similar, so very little if anything goes out of range, and I get what looks like I would expect for a detail layer (similar to a HP layer). But when I then try to grain extract the detail from the residual layer, it goes pear shaped, because the initial subtraction, before adding back the 128, sinks some (the darker ones) pixels below zero, and they then get clipped, so the added 128 starts from 0, rather than minus whatever. Doing the add 128 first just mirrors the clipping at the top end.

I think they get away with it in GIMP by using a bit of semi-voodoo, if that's not an oxymoron: during the black box calculation stage the numbers can and do go below zero, all the way down as far as -127 if needed, but they are then pulled back to 0 and above by adding 128, so what you end up seeing remains OK. So far as I can see, in PL, emulating Grain Extract (and Grain Merge) is always a 2 stage procedure: (1) the layer addition or subtraction, and (2) the adding/subtracting 128. They can be done in either order, but if at some point in one of the stages things go out of range -phut. Burkhard's neat 50% opacity then HG fix works it achieves scaled addition, A/2 + B/2, so even white in both A and B just stays in range (though I do still think I have persuaded myself the HG scaling must somehow loose some date, if only the midrange (64 to 191) of the data is being used). But there doesn't seem to be a way - at least I can't see it - to achieve scaled subtraction using a opacity and/or a blend mode. Difference looked like it might be a possibility, because it is a subtraction, but no dice.

I suppose I could go back to doing everything in 32 bits, and hope the out of range numbers work better, but I'm now wary of that because some of the images I had in mind for the technique are already large enough (typically 40MB tiffs) so blowing up the files sizes is not something to be entertained lightly...

cR

PS Burkhard - yes, having the HG as a separate layer works - thanks for the clarification

[bkh]

Burkhard's neat 50% opacity then HG fix works it achieves scaled addition, A/2 + B/2, so even white in both A and B just stays in range (though I do still think I have persuaded myself the HG scaling must somehow loose some date, if only the midrange (64 to 191) of the data is being used).

Yes, as I wrote earlier, you use one bit of precision (so this should be ok if you have 16 bits of colour depth).

Depending on your recipe, PL's high pass layer might be an option because it does something like layer - blurred layer + 128, avoiding intermediate overflows.

I suppose I could go back to doing everything in 32 bits, and hope the out of range numbers work better, but I'm now wary of that because some of the images I had in mind for the technique are already large enough (typically 40MB tiffs) so blowing up the files sizes is not something to be entertained lightly...

Maybe 16 bits is enough. Or you can switch to document mode, which allows to work in 32 bit mode even if your layers are only 8 bit, saving a lot of space. If you save the image as a TIFF, it will be 32 bits, though.

Btw., to clip 32 bit values to the 0..255 range, you can add a black layer in "lighten" mode and a white layer in "darken" mode.

Cheers

Burkhard.

[photoken]

Ray,
You might want to contact Russell Cottrell
http://www.russellcottrell.com/photo/RCFilters.htm
for some pointers about this (and your colour saturation questions, too).

He's developed some very interesting filters as Photoshop plugins, but he also tests the plugins with PhotoLine. In fact, he's posted a couple of messages in this forum.

[bkh]

Btw., if you decide to work in 32 bits, then there's a very simple way to get the Grain Merge mode in PL: just use Linear Light at 50%. (This still produces values outside the 0..255 range.)

Cheers

Burkhard.

[cathodeRay]

Ken - I have looked at RC's work with great interest - he has come up with some very clever plugins, and his inclusion of Photoline is of course very welcome. Maybe he's already following this thread!

Going back to wavelet decomposition: I have sort of got it working. I've gone with 32 bits as it seems to make things, especially the maths, simpler, and used a 128-grey layer in Linear Dodge (Add) mode to add 128, and used Burkhard's Linear Light at 50% to get the Grain Merge (it can also be done by using Linear Dodge (Add) and a 128-grey layer in Subtract mode which does make the layers easier to edit but pretty much doubles the file size). The five layers use blurs of 1, 2, 4, 8 and 16px, and at each iteration the extracted layers was 'grain extracted' from the residual layer. The final stack looks like this, and hopefully you can see that the main display matches the Original (the turned off Original layer at the top is just a check to toggle/confirm the stack is working properly):

wavelet decomposition.jpg

You can also just see the increasingly coarse details on the lower (larger pixel blur) layers, and the loss of detail (but not colour) on the Residual layer.

I have tried more time than I care to mention to record the steps as an action but either I get distracted/lose the plot halfway through and leave out something or otherwise do something I shouldn't, or PL somehow doesn't play the action back right (I think it may be how the layers get selected - sometimes I used the arrow keys to move around, other times I used the mouse to select the layer). Not for the first time: I do miss the ability to edit actions as simple scripts/text files - that way I could tidy the action up. Nor does the somewhat clunky recording/step through/quasi debugging help - but before that what would really help is the afore-mentioned ability to edit the actions as text.

cR

PS Burkhard - I meant to say: when using 32 bits it seems to me eg by watching the colour panels that at certain points the pixels values do go out of range - but isn't that what we want? The distortions come in when they get clipped so the +/- starts from 0/255 rather than from the 'true' value, even if it is nominally out of range at that point.

[photoken]

...
I have tried more time than I care to mention to record the steps as an action but either I get distracted/lose the plot halfway through and leave out something or otherwise do something I shouldn't, or PL somehow doesn't play the action back right (I think it may be how the layers get selected - sometimes I used the arrow keys to move around, other times I used the mouse to select the layer). Not for the first time: I do miss the ability to edit actions as simple scripts/text files - that way I could tidy the action up. Nor does the somewhat clunky recording/step through/quasi debugging help - but before that what would really help is the afore-mentioned ability to edit the actions as text.

Yep. I'll take this opportunity to once again repeat my long-standing request for a "real" History panel. One that allows non-linear selections, undos, and replays. With the capability to put the selected steps into an action.

Editing the actions as a text file would help, but I'm a GUI kind of guy , so I prefer the intuitiveness of the aforementioned History panel....

[cathodeRay]

Ken - it might be better to start a new thread on the matter of the History Panel and Actions, which although linked are not quite the same thing. I just brought it, ie the clunkiness of Action handling in general, up here after suffering the pain - yes, pain - of trying to record the steps to do a wavelet decomposition. As noted above, it was the pain that won in the end, so no action. It goes without saying that if better use could be made of the History Panel and Actions, notably the ability to edit Actions easily, then that would be another way in which PL differentiated itself positively from the SWTMNBN.

So - new thread coming up shortly!

cR

[cathodeRay]

FWIW, here in plain text are the steps I used to create a wavelet decomposition in PL:

EDIT 7th Nov 2017: these steps need to be followed EXACTLY as stated to work properly. I got burnt by missing out 'Turn off its visibility for now' in step two...

1. Start with a single layer file with the original image. Name the layer original, and set it to 32 bits, needed to ensure the maths behaves itself. You do not want any clipping to happen, ie the numbers must be allowed to go outside 0...1/0...255 during the calculations. At the end they are all back in range. And yes, your hard disk will get very full if you make a habit of doing this.

2. Duplicate Original twice. Name the top one Original-Chk - it will stay at the top and is used as a visual check (by toggling it's visibility on and off) that the stack is correct. Turn off its visibility for now. This means that at the end of each of your iterations below you will see the results of your work so far in the main image view (and hopefully will spot if you have got something wrong). Name the intermediate layer Residual.

If you are feeling inquisitive, you can put some colour picker points on the main image, and watch the byte numbers in the colour panels. You will see that while the main image is often bizarre in intermediate stages, the byte numbers are indeed capable going, as they should, outside the 0-255 range, and then correct at the end of each iteration.

3. For the extraction iterations, proceed as follows. I did five iterations, with blurs at 1, 2, 4, 8 and 16px:

(a) copy Original and place the copy above Residual (I usually duplicate and drag to do that).

(b) do a Gaussian blur at the current iteration pixel setting on the Original copy layer

(c) set the Original copy layer blend mode to Subtract

(d) add a new 128-grey layer above the Original copy layer, and set its blend mode to Linear Dodge (Add) (subtract and add 128 = grain extract)

(e) merge copy and paste the 128-grey layer, and name the copy Scale [n]px where [n] is the blur size you used

(f) delete the intermediate layers, so that Scale [n]px is now directly above Residual

(g) set Scale [n]px blend mode to subtract, and add a new 128-grey layer above it with its blend mode set to Linear Dodge (Add)

(h) merge copy and paste the 128-grey layer, and name it Residual (at this time there are two layers named Residual - that's OK)

(i) delete the unwanted intermediate layers, including the lower Residual layer, and place the newer Residual layer directly above the bottom Original layer the stack should now look like this:

Original-Chk
Scale 1px
Residual
Original

and after the next iteration is will look like this:

Original-Chk
Scale 1px
Scale 2px
Residual
Original

etc etc

(j) finally, either (a) set the Scale [n]px layer blend mode to Linear Light, 50% opacity or (b) set the Scale [n]px layer to Linear Dodge (Add) and add a 128-grey layer above it with blend mode set to subtract. Both equate to grain merge, the former is more economical of disk space, the latter makes editing the detail extraction layers easier, as they are at 100% opacity rather than 50%. Try it, and you will see what I am getting at!

Do the next iteration, until you have all the layers you want. No wonder the Action blew up!

Hope that helps!

cathodeRay