X-Ray, or rather IR, vision

[cathodeRay]

Back with another challenge! This time I'm restoring some 1980s Kodachrome slides shot in Africa, where only mad dogs and photographers go out in the midday sun. A lot of the slides have faded a bit but the bigger problem is the intense shadows. Eye-balling the slide itself suggests there is some detail hiding in the shadows but none of the usual tricks including increasing scan exposure by +8EV (real increase in exposure time, not in SW) etc got very far. Then I just happened to notice that the IR scan had X-Ray eyes. Being Kodachrome, it was picking up quite a bit of image data as well. It could even see the bark on the invisible tree! So, I thought, what if I use the IR scan as the luminance channel? I set the B&W points on the IR channel (the actual IR scan is very flat) and tried (a) to use it as the luminance channel (copy to clipboard, clipboard to L channel using 'Change Channels' in Lab mode) (b) put it above the background and set to luminance blend (by the book that means luminance from the top layer ie the IR scan, hue sat etc from the bottom layer), even used the Channel Mixer to change the relative contributions from each layer, but all to no avail: I just can't get the colour to shine through (methods (a) and (b) produced very similar if not identical results). Any ideas? First attachment is the ordinary scan, second the IR scan with B&W points set and the third my dismal attempts at getting the two to work together:

scan.jpg

IR scan.jpg

luminance blend.jpg

cathodeRay

[cathodeRay]

OK, a step in the right direction: use the unadjusted IR scan, luminance blend, 50% opacity:

unadjusted IR scan.jpg

50pc luminance blend unadjusted IR scan.jpg

Adding a duplicate layer in multiply blend mode = bizarre blue result, even in RGB mode...

cathodeRay

Edit: also tried adding a dodge/burn overlay layer at 100% burn = goes back to the original scan ie all shadow detail lost...

[Martin Huber]

How about adding a layer mask to your correction layer. In my example I used your corrected IR scan as correction layer, and an inverted, blurred version of it as the layer mask of the correction layer.
I used virtual copies for all that, because this way you can adjust the contrast and brightness of the background by modifying a single layer ("IR scan").

IRScan.pld.zip

Martin

[photoken]

I tried it using the IR scan as the top layer in "Soft Light" blend mode:

lion 01.jpg

Looks like a good starting point for Curves, Hue/Saturation, etc., adjustments....

[photoken]

Here's what I came up with after about 10 minutes:

lion 02.jpg

I used the Histogram adjustment layer to change the gamma to 1.60, the H/S layer to boost the saturation to 100%, and applied a soft "S" curve in the Curves layer to the "L" channel in Lab mode.

The layers look like this:

lion 02 layers.jpg

Edit:
I inadvertently screwed up the layer order before exporting the layers screenshot. The two image layers are at the bottom of the stack. Sorry about that....

[cathodeRay]

Martin and Ken - great input from both of you on what I thought was a particularly difficult slide (of many similar ones I have to work on, so well worth the effort to find a method that works). Martin's version (worth downloading to see it) has got a pleasing and coherent retention of texture both in the background and the foreground (it's a large shaggy dog, by the way, not a small lion!) and Ken's is starting to show some useful colours in the background, but perhaps at a cost to the dog's fur which has lost a bit of its richness. It certainly seems that, at least for Kodachrome, using the IR scan to capture what the visible light can't see may have its uses, though the low levels of colour info in the shadows remains tricky. I'm trying to think if there might be some way of amplifying what is there to get some more colour? The trouble with boosting saturation of a dull colour is that it just tends to become just more saturated dullness, plus it quickly starts to lose what little contrast it has. I did try some curves on the Lab a and b channels but as they are very much of a mind of their own, with small curve changes causing big colour changes, all I managed (not entirely unexpectedly) was some rather bizarre changes.

The Holy Grail for me at this stage in the restoration of an image is either (a) true colour transplantation, by which I mean injecting one known good colour palette from an image which has the known right colours into the one in need better colour, and I think - so far as I understand it - the problem is there isn't an easy way to map the injected colours to the recipient image's colours, or (b) amplification of what little info colour there is to useful levels. For me, the aim is that the image should look authentic, rather than colourised. We were up against the same problem with the photo of the yacht off Brittany that I posted a while back...

cathodeRay

[photoken]

I did some more work, following my original methodology:

DogInAfrica.jpg

(The JPG conversion for posting here darkened the shadows -- see the PLD file for a better overall impression.)
I got as much green/blue into this one as possible; you can really see the green grasses in the foreground, now.

The layers are quite a bit more complicated:

DogInAfrica layers.jpg

and instead of trying to describe all the adjustment settings, I'll just provide the PLD file:
Ooops, the zipped PLD is too large to be attached here. Oh well, you can always experiment with the adjustment layers to see what's being done.... The lower Curves layer is working in HIS mode to softly boost the less saturated colours, and the upper Curves layer is working on the "L" channel in Lab mode to apply a more aggressive "S" curve than the one I used in my first image.

I think this is about the limit for this technique. You're absolutely right about the problems with "creating" colours in the shadow areas where the dyes seem to have faded out the green/blue colours.

The next step would need to be using the IR scan as a mask for adding colours, as we did with the yacht picture, I'm thinking....

[cathodeRay]

The image presented here is, as I am sure you have all realised, about a one third crop of the original slide (cropped to limit file size as well as save the blushes of the people in the rest of the slide). On the whole slide, as here, the foreground is not too bad, mostly general fading and a minor colour shift. The major problem area is the background (well over half of the whole slide) which hardly lets through any visible light but, crucially, does reveal structural detail under IR light. My initial plan was for a two pronged approach: regular touching up of the foreground, combined with a restored background using a gradient mask to handle the transition. I think I am right in saying Martin has already started to go down that route, by using a mask as he has.

So the key thing is getting back (ie amplifying) or injecting colour into the background. The poverty of data in the background is clearly seen in the RGB histogram, which says it all: a capital L shape, with the dominant vertical representing all those dark pixels. Moving in the black and white points and setting the gamma correction to 5.0 (ouch!) has an 'interesting' effect: some washed out colour appears, but detail/contrast is very poor. Tweaking saturation makes the mud look like psychedelic mud...

So it's back to amplifying or injecting. The most likely candidate for the latter is a program called Color Pilot (http://www.colorpilot.com/pilot.html) which can 'inject' a palette from one image into another. Unfortunately it is only 8 bit, but sometimes it works wonders, but not here, or in other similar problem situations (at least, I haven't had much success). On the former (amplifying), I recently came across some python scripts originally developed a while ago as python plugins for GIMP (so also only 8 bit), but now available as standalone scripts here: http://www.lionhouse.plus.com/photosoftware/restore/. The standalone scripts are pyrestore2.py and pyrestore3.py: both are for python 2.x (the author has done a python 3.x version, and I expect may post it soon enough), with the 2 and 3 suffix referring not to the python version, but to the engine/algorithm used to do the restoration, with 2 being pythons internal routines, 3 being the author's own algorithm. The pdfs available on the site give much more information.

The aim of the scripts is very targeted: to reverse the colour fading that happens to slides, so it seems worth seeing what they do to the image. The maths is way over my head, but I think I understand the idea, which I see as if the fading can be mathematically described as A->B, then applying B->A to the faded slide should move it back towards its unfaded state. Here's what happens when the 'restore3' version of the script is run on both Martin and Ken's early version (lion 02) of the image (with the original scan for direct comparison):

scan.jpg

martin.jpg

ken.jpg

Both get more colour. Focusing on the area of interest, ie the background, Martin's has to my eye a slightly 'painterly' effect, with greens dominant, while Ken's has more colours, some of which are not a million miles from reality, and both are a huge improvement over the original. In Martin's version, the foreground is also noticeably improved. Note that neither have had any direct adjustments to saturation per se ie via a saturation slider/brush/curve. With a bit more tweaking either might pass as a good enough restoration.

cathodeRay

[photoken]

A few more thoughts on this:

If a "true" Luminance image is used, your initial method of using it as a Luminance layer/channel can work very well. See my examination of this technique when used with astrophotography images:
http://www.pl32.com/forum3/viewtopic.ph ... phy#p36796

Can you re-scan the Kodachrome using the scanning software to output the result as a B&W image? Perhaps altering the properties of the scan to get more detail in the shadows (even to the point of only using the green channel as input). If so, that B&W image would be a "true" Luminance image and could be used as you originally intended.

Also, PL 20 now has a Color Lookup function; so if you can generate a "full colour" Color Lookup Table (CLUT) from a well-exposed Kodachrome, you can try applying it to your problem images. The downside is that you will lose some of the subtle shadings....

[cathodeRay]

Some interesting thoughts, Ken. I can certainly rescan the slide in all sorts of ways (I use Vuescan, and the scanner allows changing exposure in hardware for both visible and IR light plus RGB per channel gain but I think that must be in SW). Looking at your astro example, I wondered about reversing the process: apply the colour to the luminance, rather than the other may round (although they might be mirrors of each - I forget the correct term - ie the end result is the same).

I also had a go in Color Pilot (mentioned earlier with a link) but once again found that although sometimes it works miracles (eg in the examples provided - funny that should be so!) but fails on others including the current problem image. CP's other problem is that although is allows masking (called 'layers' for colour masks and 'selections' for drawn masks - terminology games again), it is binary ie no feathering let alone grey scales, so the edges look horrible.

CLUTs: I had already started looking into these, thinking they might be the route to the Holy Grail of applying the palette from one known good image to the defective image. I even discovered RawTherapee can open non raw ordinary image files (clue: the clue certainly isn't in the name) and (a) has a useful, indeed extensive range of exposure and colour correction tools (b) has the ability to apply CLUTs. These are presented as 'Film Simulation' under colour but there is nothing to stop the Hald CLUT file doing something else eg apply selenium toning to the identity ('no-change') Hald CLUT and then apply the modified Hald CLUT to an image. Since the Hald CLUT is an ordinary image format (tiff or PNG), it can be edited in any editor.

Where it comes apart is in how to capture the palette from one image and apply that palette to another image. Looking at Hald-CLUT manipulation more closely, it is actually just another way of recording a set of changes - in other words another way of recording, as in actions, only the changes are stored as agnostic input/output pairs, rather than as discrete specific changes to whatever parameter is being changed. That means to get a Hald CLUT that will correctly colourise a faded or seriously underexposed slide you would have to know in advance all the corrections you wanted to apply! So I suppose while it may in theory be possible to set up a Hald CLUT that represented the characteristic changes required eg to correct under-exposure, I suspect in many cases the changes will to a greater of lesser extent be image specific...

Prompted by the Film Simulation approach, I had a look at NIK Analog Efex (I had previously tried CEP but not really got anywhere) and found that Analog Efex (in Basic Adjustments) actually did better than CEP (more useful sliders) but it still wasn't anywhere near right. I also tried some voodoo (multiplying inversions many times and then re-inverting) in PL and again improvements could be seen, but not yet 'good enough'.

I haven't bit the bullet and upgraded to PL20 yet, but if it really can "generate a "full colour" Color Lookup Table (CLUT) from a well-exposed Kodachrome" which can then be applied to specific problem images, then it may be the answer, though given the above thoughts on CLUTs I fear it may not be as simple as it sounds... Maybe time to download the trial and see what I can do with it, though.

Of course, that old riposte GIGO may be the underlying problem here, as in there isn't enough starting data in the slides let alone the scans. I'll see how I get on with the rescans on different settings, though as mentioned before even cranking up the EV by eye-watering amounts didn't do much except blow out everything else in the slide. GIGO...

cathodeRay

[photoken]

Ray,

No, no, no! Sorry if I was not clear, but PL20 can not generate CLUTs. It can only apply them to images.

You'd need to find a CLUT generated from a well-exposed Kodachrome and see if applying that to your problem images does anything useful.

A word of warning about CLUTs, however: at first I was really excited about gaining the ability to apply them in PL20, but after doing some more reading about CLUTs, I have become indifferent to them. As I understand the process, because it takes a whole lot of processing power (and time) to apply them to an image, their colour tables have been grossly simplified. That means that the subtle colour gradations and tonal ranges in the original image are lost. The use of CLUTs is almost exclusively in the video industry (which has to process hundreds of thousands of individual image frames), and that loss of quality is the tradeoff that industry has accepted for their purposes.

Your explanation of your scanning software's capabilities makes me think that it would be a worthwhile experiment to scan one of your Kodachromes four times: once each for the individual R, G, B and Luminance channels as B&W images; and then combine those B&W images into a full colour image as I did for the astrophotography shots. No guarantee, of course, but an interesting idea....

[cathodeRay]

Ken - yes, my original response to the apparent possibilities opened up by CLUTs was much the same, only to become more sanguine. As you say, they are used chiefly in video work, where an individual image is less than a blink of the eye and trade-offs can be usefully made. A still image is another matter. And, as I said, I don't think CLUTs can do colour injection per se, only carry out a pre-set set of transformations. It does seem there is some interpolation involved, which apparently helps smooth out transitions. It is not clear how much a bigger CLUT makes for a better transformation.

As I said, I don't think you can generate a game changing CLUT from an image - what CLUTs record is changes: delta X, if you like, rather than X.

I've an awkward feeling that my Holy Grail (what made me think it might be possible was Color/Match Color and Color Pilot's implementation) is impossible, for a number of reasons, mainly that there is no way of determining in advance how the defective pixels get mapped to the corrected pixels instructions in the CLUT. I'm sure you could do it on a per image basis, but that is a very long and time consuming prospect, with a significant prospect of losing the will to live before project completion.

Rescans are the way forward. Will report back when I have something to say...

cR

[cathodeRay]

OK, here is the rescanned version, blended with the original. I cranked up the exposure time in Vuescan until the background area started to show white clipping and then took it down one step, noting in passing that Vuescan's exposure setting is linear rather than exponential. As noted before, I am as sure as can be that this is a real hardware increase in exposure - the scans do take longer, which makes sense. I tried all sorts of different adjustments including different R G B analog gain settings, different colour setting, scrunching up the histogram to expand the vertical bar of the L shaped histogram, but none added any significant improvement given the aim, which was to get maximum detail from the dark background area (not worrying what happened to the foreground). Linear scans added nothing, in fact they made things worse given the problem we are dealing with.

I then opened the scan in RawTherapee and tweaked the exposure a bit before adding the scan as a layer above the original scan in PL and did a standard paint on layer mask to show the foreground as required (blend mode Normal but at 150% which just tightened it up a bit) to get this, which I think is 'good enough' (bar final curve tweaking and sharpening). The colour and tone blend reasonably enough and there is now real detail in the background: the sky seen through the leaves top right looks convincing enough, there is a sense of the bark on the tree and now even the wires of the fence middle left are just visible. The dark band directly above the dog's back is I think real, being the even more intensely dark shadow from a tree out of frame to the right. There might be something that could be done to lessen its impact. Original scan also reposted for direct comparison.

The lesson in all this for me is the very old one: do as much work as you possibly can in the camera/scanner...

probably good enough.jpg

scan.jpg

cathodeRay

[photoken]

I don't think that dark area above the dog's back is real. It can't be a cast shadow because it is also darker in other shadow areas. It's likely an artifact of the procedure, as are the halos around the white fence structure.

Did you try outputting the individual R, G, and B channels as B&W scans?

[photoken]

Well, I tried throwing everything in RawTherapee at this. Given the limitations of an 8-bit original, nothing in RT yielded anything useful. Not Retinex, not Wavelets, nothing.

So I kept coming back to the simple technique of using the IR scan as a Soft Light blended layer. My latest result:

DogInAfrica 02 w.jpg

is, I think, very good. The overall balance between shadows and sunlight is nice, the shadowed background is distinctly bluish, the texture of the tree bark is there, the fence wires are visible, the background has a variety of colours, etc. (Again, the PLD image shows more detail in the background than does this JPG generated for forum posting.)

The layers look like this:

DogInAfrica 02 w layers.jpg

The H/S adjustment boosts "All" by 100%, the "Greens" by 87%, and the "Cyans" by 50%. The Hue Editor adjustment is used in "HSV" mode to boost saturation of the greens and blues while holding the saturation of the reds to hardly any boost at all. The Color Balance adjustment shifts the Cyan/Red balance to -18 and the Yellow/Blue balance to +8. The Curves adjustment is working in Lab mode, with a gamma of 0.95 and an "S" curve on the "L" channel to slightly lighten the darks and significantly darken the mid-to-lights.