Fun with Scripting - Wavelet Decompose

Here everybody can post his problems with PhotoLine
cathodeRay
Mitglied
Posts: 151
Joined: Sun 15 Nov 2015 12:37

Fun with Scripting - Wavelet Decompose

Post by cathodeRay » Sat 22 Feb 2020 12:07

Hi All

Back again after a break from posting, and now trying to get to grips with scripting. This is something I was very pleased to see implemented, as I have always struggled somewhat with the opacity of actions. Scripting not only allows direct editing of code, unlike actions, which are always in effect recordings, but it also opens up the possibilities of things like loops and conditionals. I waited a bit before jumping in to scripting, to let things mature a bit. I'm using the latest main release version, v21.50 (Win 64bit), not a later beta version.

Now, as we all know only too well, PL may be brilliant and extremely powerful and capable, but documentation isn't its strong point, and this holds true for scripting as well. Russell has done some excellent work, and generously made it available on line. PL does come with a PDF detailing many of the scripting interfaces objects properties and and methods but it is very thin on basics. There are also a few posts here in the forum that touch on various ways of doing things. However there are two (I think probably related) things I am struggling with at the moment (plus a couple of smaller points).

The first is how to access layers easily and consistently. Let's say we want to duplicate the third from the bottom layer in a five layer stack. The layer as yet doesn't have a name, either in the script or visible in the layer stack. Now I think I am write in saying the whole layer stack is a tree, with its root hidden from view, with the consequence that the bottom background layer is actually a (first) child (again, confusing, the visible 'root' is actually a child, but we can live with that). From what I have read, the stack is also an array, or treated as an array, because zero or one based identifiers are used (as in myArray(3) might be the third or forth item in an array, depending on whether zero or one based counting is being used).

Now, if we know that the layer of interest (the third from the bottom) is the active layer (which we don't) then at it's simplest this works (and reliably so if there is only one background layer):

Set aLayer = doc.ActivePage.ActiveLayer
Set aLayer = aLayer.duplicate

because duplicate returns another layer which has now become aLayer (but that is the name of the object, not the stack visible name of the layer)

but it is messy and ambiguous. Maybe the second line should be bLayer...

Elsewhere on the forum I have read that the layers can be accessed using zero based doc.RootLayer(i) or doc.RootLayer.Item(i) (so the third layer from the bottom will be doc.RootLayer(2) or doc.RootLayer.Item(2)) and additionally the first and last layers are doc.RootLayer.First and .Last and furthermore we can also count down from the top, so in a five layer stack (with layers 0 1 2 3 and 4) and a count of five, layer 3 with be (I think) doc.RootLayer.Count - 2 (but it might be - 3! - layer IDs go from 0 to 4, count goes from 1 to 5).

All this seems unnecessarily complicated (eg why have doc.RootLayer(i) and doc.RootLayer.Item(i) - is there any difference? And is so, what?) so my
first question is what is the best way to access layers when they only have a place in the stack and no name (as yet).

Which brings me on to my second question (which is really a sub-question of my first question): how to reliably know the 'handle' of a merge copy and paste layer. First off, this does work (dupe already has been set using Set dupe = 'something that works'):

Set dupe2 = dupe.duplicate
dupe2.Name = "Name of this Layer" (this becomes the visible name in the stack, not the name used in the script)

and then maybe we do (blurRad has been set using an input box)

call dupe2.DoOperation("GaussianBlur", "Radius", blurRad)

or (and again not sure which of these two ways is better/more correct or maybe it doesn't matter)

dupe2.DoOperation "GaussianBlur", "Radius", blurRad

So the visibly named "Name of this Layer" layer gets a blurRad pixel blur. So far, so good.

Now lets say we want to do merge copy/paste. This is not merge down (= one layer after doing it), but merge copying a particular layer (maybe it has a blend mode set, so we need to 'see' the underlying layer), and pasting (inserting) it above the top layer of the current merge copy set of layers, so the end result is one more layer than before doing it (Ctrl + Shft + C then Ctrl + V if using the keyboard normally). We can't use layer.duplicate, as it only duplicates the current layer (there is no merge).

doc.merge does merge down (result is only layer, with all the above ones merged down into it)

doc.copy
doc.paste

seems to work but while doc.paste sort of makes sense ('paste into the document at the current location' but it could also mean past a new document...), doc.copy seems ambiguous - are we copying the current (active) layer (if so why doc.copy? Why not layer.copy, which doesn't seems to work/work reliably), a merged copy of the active layer and those it interacts with, or a copy of the whole document (which is what the semantics imply?). Why not layer.copy (there doesn't seem to be an IPLLayer copy method, copy is only available for IPLDocument)?

So the second question is how to do merge copy/paste, and reliably know how to access the pasted layer (assuming it is a layer, as it seems to be, even if we are applying methods to the document)? It seems possible that doc.copy followed by doc.paste might do it (a layer appears which looks like it is a merged copy) but the semantics are vague and ambiguous. And furthermore, the pasted layer doesn't appear to have a 'handle' beyond its number in the stack (doc.RootLayer(i)), which is transient and variable as other layers get deleted added and/or moved.

I tried

Set dupe3 = doc.paste

but it appears dupe3 doesn't get set properly (it seems to exist, but nothing can be done to it and MsgBox "Layer Type:" & dupe3.Type gets

Object doesn't support this property or method: 'dupe3.Type' (suggests the object exists, but can't be accessed)

Put another way, is there a way to do Set dupe3 = [object created by doc.paste] or can it only ever be accessed via its number in doc.RootLayer(i)?

BTB - a crude way of getting stepping for debugging is to add MsgBox "Pause (plus a number if it helps)" as quasi-breakpoints - clunky but it works.

BTB 2 - it seems the easiest way to add a new layer with a solid colour is to duplicate an existing layer and then do ReplaceColor:

Set dupe2 = dupe.duplicate ' better/neater than creating new layer from scratch
call dupe2.DoOperation("ReplaceColor", "DestinationColor", Array(0.5, 0.5, 0.5), "DestinationTolerance", 0, "SourceColor", Array(0.5, 0.5, 0.5), "SourceTolerance", 1)

This uses a wide source tolerance to make the duplicated layer mid grey (and those in the know may by now have guessed what this script is for (my old buddy Wavelets Decomposition) and why I am so pleased we now have loops - perfect for this sort of iterative work).

To keep the numbers as floats (so we can go outside 0 - 1 without errors) and to keep the file size down I have set the stack to Document mode

doc.DocumentMode = true

but to set the stack to 32 bit (to get the floats but without having to set each layer to 32 bit) I have to use an Action (doesn't seem to be a native way to do this in scripting) via

Dim base64Str

base64Str = [the base64Str copied from the action]

call doc.DoOperation("Action", "Data", base64Str) (thanks again to Russell for showing us how to do this)

which (I think) means the script is no longer self contained - or is it? Is that base64str in the script actually the action, all or it, or a finger pointing to the action? Experiments suggest it is the entire action, not the pointing finger, because if I delete the action, the script still works. So the script is self contained -

One other point: a script such as the one I am working on with may steps causes an Exploding Undo panel, with many of the entries blank. Is there a way to turn this behaviour off (I can't find one) and/or have just one entry eg WaveletDecompose if that is what the script is called?

I will of course upload the script once (if...) I get it working.

Thanks in advance for any help (and sorry for such a long post)!

cathodeRay
Last edited by cathodeRay on Sun 23 Feb 2020 16:52, edited 1 time in total.

User avatar
russellcottrell
Mitglied
Posts: 175
Joined: Sat 26 Jul 2014 10:13

Re: Fun with Scripting

Post by russellcottrell » Sat 22 Feb 2020 18:41

Layer references frequently require brute force because layer objects are not permanent; if you paste, resize, rotate, etc., the old layer gets taken out of the tree and you have to start over; new layers have no inherent “handle”. It is frustrating. See http://www.pl32.com/forum3/viewtopic.php?f=1&t=6096

The Base64 string in a DoOperation is run directly, meaning it is self-contained as you described; it is not a pointer to another action. It can be edited and so forth like any other string, and there does not need to be an actual action corresponding to it.

cathodeRay
Mitglied
Posts: 151
Joined: Sun 15 Nov 2015 12:37

Re: Fun with Scripting

Post by cathodeRay » Sat 22 Feb 2020 22:06

Hi Russell

Thanks for the reply. I agree there seems to be little alternative to brute force (and some quasi-hackery) to get the layers to behave. You will see from the attached script how I have dealt with it (mostly by careful position in the stack counting). The trouble with this rather than having a real name/handle is keeping track of where everything is. I also managed to confirm that, as you already knew, once in the script the Action exists independently in the script, making it self contained, and the Action can be given the Deep Six.

The other thing I discovered (so part answering my earlier question) is that .merge can take an array of the layers to be merged, and, unlike in the normal merge copy/insert paste operation, it deletes the donors ie those layers in the array. This is a double edged sword, depending on whether you still need some or all of the layers (park a copy out of harm's way) and/or need to delete some or all of them (one less step to do).

The attached zipped script is working here (PL 21.50, Win 7 64bit) and covers the essentials, though has some polishing to be done, but compared to manual wavelet decomposition is eye wateringly fast (and painless). It is set up to work on any single flat one layer image (that is important, as it uses layer count position to know what's what, if there are other layers it will probably explode your image), jpg, tiff etc, of 8 or 16 bit depth, though the maths is done at 32 bits to avoid clipping, done by using Document Mode (which you can set back, via the Document panel, to Picture mode afterwards). For those who just want to use it, you will get a message box/input boxes explaining what to do (set how many layers in the decomposition stack, and what the starting blur radius should be). At the moment, the Scale (retouch) layers are at 50% opacity (see post passim for why this is) making them rather washed out - I may change the way the final blend is done. You touch up by painting in 50% grey on the relevant Scale layer.

The top Orig-Check layer (just a copy of the original image, default visibility off) is just there as a quick visual check the decomp has happened as it should (no visible difference between it and the rest of the layer stack)

For those interested in scripting, I have left quite a few comments in the script to show my thinking, and how I tackled the various hurdles, and the rest should I hope be obvious from the script itself. Any comments/thoughts gratefully received!

cR

WaveletDecompose.zip
You do not have the required permissions to view the files attached to this post.

cathodeRay
Mitglied
Posts: 151
Joined: Sun 15 Nov 2015 12:37

Re: Fun with Scripting

Post by cathodeRay » Sun 23 Feb 2020 16:50

Some further thoughts/questions as I hone the script:

(1) I am going to add an initial check that the image is a flat one layer picture, and also reset the mode to Picture mode at the end.

(2) I would like to use, or at least have the option of using, the Linear dodge (Add) then subtract 50% grey as the way of doing the grain merge, as it leaves the Scale layers at 100% intensity, so they easier to see. It is perfectly doable but adds a lot of bloat both layers and bytes (and steps, but less of a problem as the script does them). So I have tried various ways of using an adjustment layer including the use of a custom filter to do the maths but so far no cigar. Curves and HG Correction fail because the always (curves) and usually (HG Correction) clip even though floats are being used, and the HDR HG doesn't appear to work, although it can handle the out of 0-1 range numbers. Along the way I am also getting myself confused by what happens when say the add adds 0.2 + 0.2 and then the minus 50% subtracts 0.5 so the float is correctly -0.1 but then surely that must result in clipping in the final result - which doesn't seem to happen...

(3) Retouching is currently destructive, and I would like to make it non-destructive and have tried various options. In essence what is need is to be able to cover/paint the non-mid grey areas in the scale layers with mid grey. I have tried adding a new white layer in the stack above relevant scale layer and tied using a colour filter (blend if) to allow mid grey painted on that white layer to show though but it doesn't work or I am not using the right settings. I have tried adding a similar layer with 0% opacity, and tried to find a way to paint using opacity - but I think opacity is a global setting for the layer (hence try to use a colour filter to get round that). I have tried using masks but they, ahem, mask, ie make transparent (or opaque), rather than make grey. Is there a way to use an adjustment layer to paint grey on to the adjustment layer's parent? I can't see one, but maybe there is.

Yes, with the script being so quick and easy to use one can always reset the stack to 'as new' if ones messes up with destructive editing, but that may mean losing previous edits that were OK and so will have to be redone.

Any thoughts very welcome!

cR

Martin Huber
Entwickler
Entwickler
Posts: 3653
Joined: Tue 19 Nov 2002 15:49

Re: Fun with Scripting - Wavelet Decompose

Post by Martin Huber » Fri 28 Feb 2020 17:15

cathodeRay wrote:
Sat 22 Feb 2020 12:07
Elsewhere on the forum I have read that the layers can be accessed using zero based doc.RootLayer(i) or doc.RootLayer.Item(i) (so the third layer from the bottom will be doc.RootLayer(2) or doc.RootLayer.Item(2)) and additionally the first and last layers are doc.RootLayer.First and .Last and furthermore we can also count down from the top, so in a five layer stack (with layers 0 1 2 3 and 4) and a count of five, layer 3 with be (I think) doc.RootLayer.Count - 2 (but it might be - 3! - layer IDs go from 0 to 4, count goes from 1 to 5).
There are no layer IDs. A layer allows accessing its children as an array and the array indices go from 0 to (Count - 1).
cathodeRay wrote:
Sat 22 Feb 2020 12:07
All this seems unnecessarily complicated (eg why have doc.RootLayer(i) and doc.RootLayer.Item(i) - is there any difference? And is so, what?)
That's the way Windows automation works. A layer acts as an array and in Windows automation an array has to implement Count and Item. VBScript simplifies accessing the array by allowing to skip writing ".Item".
cathodeRay wrote:
Sat 22 Feb 2020 12:07
so my first question is what is the best way to access layers when they only have a place in the stack and no name (as yet).
This always depends on what you want to achieve.
cathodeRay wrote:
Sat 22 Feb 2020 12:07
Which brings me on to my second question (which is really a sub-question of my first question): how to reliably know the 'handle' of a merge copy and paste layer.
You mean the Merge and the Paste operations of IPLDocument? Both return their result:

Set mergedImage = aDocument.Merge
Set pastedLayers = aDocument.Paste

mergedImage is an image layer, pastedLayers is an array of layers.
cathodeRay wrote:
Sat 22 Feb 2020 12:07
First off, this does work (dupe already has been set using Set dupe = 'something that works'):

Set dupe2 = dupe.duplicate
dupe2.Name = "Name of this Layer" (this becomes the visible name in the stack, not the name used in the script)

and then maybe we do (blurRad has been set using an input box)

call dupe2.DoOperation("GaussianBlur", "Radius", blurRad)

or (and again not sure which of these two ways is better/more correct or maybe it doesn't matter)

dupe2.DoOperation "GaussianBlur", "Radius", blurRad

So the visibly named "Name of this Layer" layer gets a blurRad pixel blur. So far, so good.

Now lets say we want to do merge copy/paste. This is not merge down (= one layer after doing it), but merge copying a particular layer (maybe it has a blend mode set, so we need to 'see' the underlying layer), and pasting (inserting) it above the top layer of the current merge copy set of layers, so the end result is one more layer than before doing it (Ctrl + Shft + C then Ctrl + V if using the keyboard normally). We can't use layer.duplicate, as it only duplicates the current layer (there is no merge).

doc.merge does merge down (result is only layer, with all the above ones merged down into it)

doc.copy
doc.paste

seems to work but while doc.paste sort of makes sense ('paste into the document at the current location' but it could also mean past a new document...), doc.copy seems ambiguous - are we copying the current (active) layer (if so why doc.copy?
doc.copy copies the content of the Document (the active page) merged to a single image layer. There is also page.Copy (copy merged content of Page), layer.copy (copy the Layer, the same as selecting a layer and pressing Ctrl+C) and layerArray.copy (copy a LayerArray, the same as selecting multiple layers and pressing Ctrl+C).
cathodeRay wrote:
Sat 22 Feb 2020 12:07
Why not layer.copy, which doesn't seems to work/work reliably),
Please provide a sample script that doesn't work reliable.
cathodeRay wrote:
Sat 22 Feb 2020 12:07
a merged copy of the active layer and those it interacts with, or a copy of the whole document (which is what the semantics imply?).
The documentation of IPLDocument.Copy says: "Copy a merged image of the document to the clipboard." What is missing here.
cathodeRay wrote:
Sat 22 Feb 2020 12:07
Why not layer.copy (there doesn't seem to be an IPLLayer copy method, copy is only available for IPLDocument)?
There is an IPLLayer.Copy and it copies the layer to the clipboard.
cathodeRay wrote:
Sat 22 Feb 2020 12:07
So the second question is how to do merge copy/paste, and reliably know how to access the pasted layer (assuming it is a layer, as it seems to be, even if we are applying methods to the document)? It seems possible that doc.copy followed by doc.paste might do it (a layer appears which looks like it is a merged copy) but the semantics are vague and ambiguous.
It is really hard for me to follow your thoughts with all the interspersed questions and comments.

But if I understand you correctly, doc.Copy is the correct command. doc.Paste might be correct, too, depending on your intentions.
I don't understand what's vague and ambiguous. The documentation of doc.Paste says that it will insert the pasted layers after the selected layers and that it will return the pasted layers as result. So maybe IPLLayer.Paste might be the better command which optionally has a parameter where to insert the new layers.
cathodeRay wrote:
Sat 22 Feb 2020 12:07
And furthermore, the pasted layer doesn't appear to have a 'handle' beyond its number in the stack (doc.RootLayer(i)), which is transient and variable as other layers get deleted added and/or moved.

I tried

Set dupe3 = doc.paste

but it appears dupe3 doesn't get set properly (it seems to exist, but nothing can be done to it and MsgBox "Layer Type:" & dupe3.Type gets

Object doesn't support this property or method: 'dupe3.Type' (suggests the object exists, but can't be accessed)

Put another way, is there a way to do Set dupe3 = [object created by doc.paste] or can it only ever be accessed via its number in doc.RootLayer(i)?
Paste returns a LayerArray, so this should work:

Code: Select all

Set pastedLayers = doc.Paste
if pastedLayers.Count > 0 Then
    Set firstPastedLayer = pastedLayers(0)
End If
cathodeRay wrote:
Sat 22 Feb 2020 12:07
BTB - a crude way of getting stepping for debugging is to add MsgBox "Pause (plus a number if it helps)" as quasi-breakpoints - clunky but it works.

BTB 2 - it seems the easiest way to add a new layer with a solid colour is to duplicate an existing layer and then do ReplaceColor:

Set dupe2 = dupe.duplicate ' better/neater than creating new layer from scratch
call dupe2.DoOperation("ReplaceColor", "DestinationColor", Array(0.5, 0.5, 0.5), "DestinationTolerance", 0, "SourceColor", Array(0.5, 0.5, 0.5), "SourceTolerance", 1)

This uses a wide source tolerance to make the duplicated layer mid grey (and those in the know may by now have guessed what this script is for (my old buddy Wavelets Decomposition) and why I am so pleased we now have loops - perfect for this sort of iterative work).

To keep the numbers as floats (so we can go outside 0 - 1 without errors) and to keep the file size down I have set the stack to Document mode

doc.DocumentMode = true

but to set the stack to 32 bit (to get the floats but without having to set each layer to 32 bit) I have to use an Action
Yes, that one is missing.
cathodeRay wrote:
Sat 22 Feb 2020 12:07
(doesn't seem to be a native way to do this in scripting) via

Dim base64Str

base64Str = [the base64Str copied from the action]

call doc.DoOperation("Action", "Data", base64Str) (thanks again to Russell for showing us how to do this)

which (I think) means the script is no longer self contained - or is it? Is that base64str in the script actually the action, all or it, or a finger pointing to the action? Experiments suggest it is the entire action, not the pointing finger, because if I delete the action, the script still works. So the script is self contained -
It is self-contained.

cathodeRay wrote:
Sat 22 Feb 2020 12:07
One other point: a script such as the one I am working on with may steps causes an Exploding Undo panel, with many of the entries blank. Is there a way to turn this behaviour off (I can't find one) and/or have just one entry eg WaveletDecompose if that is what the script is called?
Undo is a problem. If a script is started from the Explorer, PhotoLine doesn't know when a script ends, so it can't group the undo to a single entry.

Martin

cathodeRay
Mitglied
Posts: 151
Joined: Sun 15 Nov 2015 12:37

Re: Fun with Scripting - Wavelet Decompose

Post by cathodeRay » Fri 28 Feb 2020 21:30

Dear Martin

Thanks for your detailed reply, very helpful as always, and you are right, I did ramble all over the place, you had every right to be tetchy, and I apologise.

You may also have seen in the first version of the script I managed to find a way of reliably identifying layers (position in arrays). Once I had got my head round the fact there is a hidden root, and the original single background layer is actually a child of the hidden root, it all became much clearer.

I also managed to get my head round the various merge options and methods etc.

Here now is version 2, with some of the personal wish list items added. Non-destructive adjustment layers (actually they are masks) now added, plus a check on the starting point being a one layer image. I have also forced the 32 bit stack images to the same bit depth as the original image to avoid 32 bit bloat.

I don't see an easy way to make the scale layers more 'readable' short of changing the blend mode back and forth - unnecessarily tedious, so I'm stuck with it as it is (unless any one has an idea how to get the scale layers to look more 'normal').

It also seems the document has to stay in Document mode, even after all editing is completed, to keep the maths working. I'm not sure if there is any downside to this.

Once the stack is set up, use Alt + Click on the eye of each scale layer to see where the various details have ended up, and then make all visible except Orig-Check and use mid grey (brush or air-brush, 50-75 % intensity work for me) to paint on the relevant mask layer to mask out the blemishes.

It seems - I may be wrong - that the apparently transparent areas on the scale px layers (after touching up) are treated as neutral grey - maybe their appearance in the stack is an artefact.

Here is a public image domain (from Pat David - https://patdavid.net/2013/03/the-open-s ... ssing.html - an excellent resource) showing what can be done in just a couple of minutes:

Before-After.jpg

It's pretty subtle, but it's meant to be - that's the whole point! I tried to attach a .pld with the before/after in a stack, making it easier to see the effects of the retouching, but came up against the file too large error.

cathodeRay

WaveletDecompose-v2.zip
You do not have the required permissions to view the files attached to this post.