Improvements and ideas for RAW workflow in PhotoLine

Hier diskutieren die Betatester von PhotoLine untereinander und mit den Entwicklern
User avatar
shijan
Mitglied
Posts: 1194
Joined: Mon 23 Dec 2019 15:21
Location: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Post by shijan »

Seems PhotoLine can open DNG files with 3:1 and 5:1 compression. This is rather unique option. DNG compression like this used only in some Blackmagic Design cameras and most RAW editing apps can't read files like these at all.
DNG files from these cameras are very special. They designed specially for DaVinci Resolve editor, and so raw data encoded in camera to 12 bit Log instead of 14 or 16 bit Linear. Also seems these DNGs contain incomplete metadata (or it is just missed for purpose), so colors are represented as BMDFilm Wide Gamut and don't transformed correctly to working color space. These files require manual calibration and custom made input ICC profile or input LUT. Those compressed DNGs behave the same "incorrectly" in other apps as well.
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
arnold_s
Mitglied
Posts: 66
Joined: Sun 04 Oct 2009 02:36

Re: Improvements and ideas for RAW workflow in PhotoLine

Post by arnold_s »

shijan wrote: Sat 09 Jan 2021 20:31 I guess problems with highlights clipping also may be just because PhotoLine use too many points for curve. In other raw editing apps those camera curves are way more simplified.
For example here is how same curve look in darktable app and PhotoLine
ImageImage
No answer up to now, so I try.
It is not the number of points, that matters for posterization. two horizontally-connected points preceeding a vertically connected point in a relevant distance building a stair in the curve could be sufficient to create posterization by decreasing color depth at least in an area. The same for highlight clipping (special cases: a stair on the right or the left border of the curve resuling in a peak at the beginning or the end of the result histogramm)
Also curve adjustment layers normally do no demosaicing (it's the process creating result colors from mixing colors of a bayer (or different, e.g. from x-trans-) sensor color filter.
Looking at your curve
Clipping: you could have meant an example not chown, the picture part chown at least reveals no clipping.
Regarding the curve, there ist no real highlight clipping, but the last part on the right side righthand from the last peek is not steep, so with low bitdepth, made stronger e.g. by gamma corrections, it could lead to visual clipping.
The same for the stair in the middle, which ist the first real difference compared to the darktable curve.
I would try removing the points in the stair and look, if the grey part recovers.
Normally you adjust curves in a way, that the curve is steeper in parts with peaks in a histogram, if that retains the natural look.
(That way contrast ist increased in parts, where you can see it, whether in parts with fewer pixels the decreased depth catches only a few eyes.)
The darktable curve respects the first big peak, whether the pl curve is - apart from the stair (repecting the last peak (?)) - nearly linear in the area containing relevant counts, which generates an effect similar to tonal correction without contrast weighting.
Strange choices, especially in combination.
User avatar
shijan
Mitglied
Posts: 1194
Joined: Mon 23 Dec 2019 15:21
Location: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Post by shijan »

Yep, shifted curve points cause the problem. I also noticed that some cameras generate normal curve with less amount of points. Here is example from Fujifilm X-T4. Seems like this thing is still under early testing:
Image
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
User avatar
shijan
Mitglied
Posts: 1194
Joined: Mon 23 Dec 2019 15:21
Location: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Post by shijan »

Most modern RAW editors offer Anti Aliasing filtration. I guess in most cases those AA filters are just a Chroma Noise Reduction masked with High Pass filter, so i guess it is more-less possible to emulate it with current PL tools. But anyway, i guess special Anti Aliasing filter in ProtoLine filters list may be very useful.
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
User avatar
shijan
Mitglied
Posts: 1194
Joined: Mon 23 Dec 2019 15:21
Location: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Post by shijan »

Here are some quick suggestions that may help to improve RAW workflow. Nothing special here. Currently all this could be done manually and recorded to single click action. I guess the only impossible yet task is to read and draw original Base Contrast Curve written in RAW file. I still have no idea if that curve actually provided inside RAW file by camera manufacturers or if it is some sort of custom-made Base Contrast Curve designed by developers of raw processing apps.
Base Contrast Curve also could be shaped manually in less precise way with combination of Luminance + Contrast (or Gamma + Contrast) sliders in Exposure tool.

So here are the ideas:

If ProPhotoRGB set in PhotoLine RAW File Preferences, ProPhotoRGB with Linear Gamma automatically should be set to debayered source raw image layer. (If possible i also would like to see ACES color space option in RAW File Preferences)
Transformation from Source Linear Gamma to Document Gamma Corrected ICC profile done with normal Color Management instead of current strange way with Gamma slider in Levels.
In this scenario any working ICC profile could be set to color managed Document or to color managed Group in "Draw Isolated" mode.
Also as you may know in PhotoLine "Document Mode" if switch Document to 32 bit, it is possible to do all processing in 32 bit, but keep original image Layer in 16 bit and so don't increase PLD file size.
In same way also possible to use "File Based Placeholder" workflow for RAW. In this scenario PLD will read external linked RAW file and so PLD size will be only few kilobytes in size.

Instead of original legacy ProPhoto g1.8 working ICC profile it could be introduced more modern variation of ProPhotoRGB primaries with sRGB or with L* TRC. Some modern raw editors also use ACES primaries with sRGB TRC as default working color space.

Used normal (non combined) Adjustment Layers tree. Everything placed inside Group. Group use name of the original RAW file. Easy to collapse into single compact layer.

Fixed Base Contrast Curve that don't auto changes randomly depending of image brightness and don't damage image look. I don't know yet if this curve should be used in Linear Gamma or in Gamma Corrected space by specification.

Instead of Levels slider, used "Exposure" slider. Easier to understand, easier to adjust exposure in f-stops. "Exposure" tool normally should be used in Linear Gamma, but i didn't test yet how Exposure tool depends of gamma in PhotoLine.

"Channel Mixer" renamed to easier to understand name "Input Color Matrix"

Image
Last edited by shijan on Tue 14 Dec 2021 23:54, edited 15 times in total.
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
User avatar
shijan
Mitglied
Posts: 1194
Joined: Mon 23 Dec 2019 15:21
Location: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Post by shijan »

Another big question is how to adjust White Balance. It looks like PhotoLine can operate only with already debayered image, so real RAW WB adjustment is not available.
White Balance could be adjusted in Linear gamma in debayered images in same way as in RAW, but from my personal tests with other software large WB adjustments in non RAW files may generates some artifacts (see details here https://forum.blackmagicdesign.com/view ... 03#p631063) But all this may depend a lot of special tool design. Need to do the same tests in PhotoLine...

For Temperature/Tint adjustment tool with a little bit confused name "Color Correction" could be used. Adjust sliders manually or select grey point with Picker!
To pick auto WB could be also used "White Point" tool in "Set Grey Point" mode and with disabled "Fix White Point" or Channel Mixer with picker. There is also "Color Temperature" filter. It needs to be tested which one from these tools produce more correct WB adjustment.
UPD. "Color Correction" works somehow strange in Linear gamma. Not sure if this is a bug or some processing limitation in this filter.
"White Point" and "Color Temperature" are work as expected in Linear gamma.

Here are also some small improvement ideas for "Color Correction" filter UI:
1. Place Blue/Yellow (Temperature) at top and Green/Red (Tint) at bottom. This is the default arrangement of these sliders used in any existing image or video editing app.
2. Add user friendly "Temperature" and "Tint" hints to slider names.
Example sketch:
Image
Last edited by shijan on Thu 16 Dec 2021 00:56, edited 5 times in total.
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
User avatar
shijan
Mitglied
Posts: 1194
Joined: Mon 23 Dec 2019 15:21
Location: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Post by shijan »

It is also very interesting to see that in some other raw editors Base Contrast Curve is not a fixed option, but organized as camera presets. So it could be changed by user to simulate different in-camera JPEG looks. https://docs.darktable.org/usermanual/3 ... ase-curve/
" base curve
Simulate the in-camera JPEG by applying a characteristic base curve to the image.
darktable comes with a number of base curve presets that attempt to mimic the curves of various camera manufacturers. These presets are automatically applied according to the manufacturer ID found in the image’s Exif data. Camera-specific base curve presets are also available for some camera models.
This module will be enabled by default if preferences > processing > auto-apply pixel workflow defaults is set to “display-referred”. A second option in the preferences dialog allows you to choose whether darktable should attempt to apply a camera-specific base curve (if found) or the generic manufacturer one."


I guess in PhotoLine this could be organized as curve presets as well. It should be some way to extract this curve data somehow...
Image
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
User avatar
shijan
Mitglied
Posts: 1194
Joined: Mon 23 Dec 2019 15:21
Location: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Post by shijan »

So for some further tests i decide to copy Base Curves from darktable using Hald cLUT pattern and convert them to 1D LUTs in .cube format. From subjective tests those curves where designed to work in Linear gamma and should be applied after Exposure adjustments (please confirm or deny this guess). Things works now same as in any normal RAW editor. I can Exposure in f-stops and see nice looking starting point without any additional manual guesses. Same time i can always turn off Base Curve adjustment layer and see original untouched "flat" source image look.
Note that it is not so easy to draw similar curve manually. Curves correction in Linear gamma needs high precision and even tiny wrong manual movement may affect image a lot. Curves like these usually generated based on calibration charts but not drawn manually by guess.
So this is how layers tree looks now:
Image

And side by side compare to default PhotoLine RAW "auto curve" look (RAW example from Fujifilm X-T2 camera from https://www.dpreview.com/reviews/image-comparison)
Image
Image
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
User avatar
shijan
Mitglied
Posts: 1194
Joined: Mon 23 Dec 2019 15:21
Location: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Post by shijan »

:arrow: So it is time to test White Balance tools in PhotoLine.
I exported two versions of same RAW image to 16 bit TIFF in ACES color space with Linear Gamma.
One image was White Balanced correctly in RAW editor before export. White Balance in another image version was adjusted in maximum possible wrong way. Note that due huge size of ACES color space color channels are not clipped after these extreme adjustments. This is important.
Base Curve and Noise Reduction where turned off in RAW editor to provide untouched "flat" TIFF output.
Image
Image

Image Layer set to ACES Color Space and Gamma 1.0 (Linear Gamma).
Document set to ACES Color Space with L* TRC.
Everything works in same way as in my RAW setup explained earlier:
Color Management transforms from 16 bit Linear Gamma Image Layer to 32 bit Gamma Corrected Document (ACES Color Space with L* TRC, or any other ICC profile that you prefer to use).
Adjustment Layers applied inside Image Layer operate in ACES Color Space and Gamma 1.0 (Linear Gamma Space)
Adjustment Layers applied on top of Image Layer operate in ACES Color Space with L* TRC (Gamma Corrected Space):
Image

Manual Curve applied on top of everything just to make look correction results more visible:
Image

Test results:
1. "Color Temperature" tool can work in both Linear Gamma Space and in Gamma Corrected space, but it don't have picker and "Tint" option, so it is useless in this situation.

2. "White Point" tool can work in both Linear Gamma Space and in Gamma Corrected space.
In "Setup Grey Point" mode and with disabled "Fix White Point" option in Linear gamma it produce perfect White Balance correction without any additional color casts, when picked grey color patch.
Image

Image

As expected in Gamma Corrected space correction is not so perfect:
Image

Image

Compare to WB adjusted directly in RAW editor i didn't notice any artifacts under deep magnification, but i noticed small difference in some colors. Both Iridient Developer and PhotoLine use the same LCMS color management system in Preferences. This difference in color could be because RAW editors adjust WB inside original camera input color space, before transformation to ACES or any other working color space.
Or this difference could be because DCP-based dual illuminant input profile in RAW editor adjusts color matrix coordinates depending of WB adjustment, but baked TIFF image use basic ICC-based input profile that can't react to WB adjustments in same way as DCP-based input profile:
Image

3. "Channel Mixer" tool can work in both Linear Gamma Space and in Gamma Corrected space. It produce near the same result as "White Point" tool when picked grey color patch.
In Linear gamma it produce perfect White Balance correction without any additional color casts.
Image

In Gamma Corrected space correction is not so perfect:
Image

4. "Color Correction" tool seems like broken or simply don't designed for White Balance correction at all.
In Linear Gamma Space it became extremely sensitive, so tiny movements of the slider shift extremely huge amount of color
:arrow: UPD. Developers noticed me that this is really a bug. Problem fixed in 23.40b2, it works normal in Linear gamma now. But seems "Color Correction" is not designed for real WB adjustments and even in Linear Gamma it can't adjust image exactly back to original:
Image

In Gamma Corrected space it also produce some color cast when picked grey color patch:
Image

And updated layout chart:
Image
Last edited by shijan on Wed 29 Dec 2021 20:36, edited 7 times in total.
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
User avatar
shijan
Mitglied
Posts: 1194
Joined: Mon 23 Dec 2019 15:21
Location: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Post by shijan »

It is also appears that Base Curve concept is totally different between different apps:

ACR, Irident Developer and darktable apply Base Curve in Linear Gamma. That curve use fixed shape and matched to specific camera model, so you need manually adjust exposure to fit dynamic range of the image inside that curve limits. This concept needs additional manual adjustment but produce more stable and predictable results.

PhotoLine, as well as RawTherapee uses automatically generated curve (Auto Matched Tone Curve) applied in Gamma Corrected Space. This curve attempt to clip some empty highlights zone if exposure is too low. As a side effect, in some complicated images it may generate artifacts and unwanted clipping instead of proper auto correction. Also the tonal look of the image will change if you switch to different working color space with different gamma.
Image
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
User avatar
shijan
Mitglied
Posts: 1194
Joined: Mon 23 Dec 2019 15:21
Location: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Post by shijan »

Another important option for RAW module in PhotoLine could be support for .DCP input profiles. There are plenty of apps and tools that support this format. But currently in PhotoLine it is only possible set custom input profile for camera in ICC format.
Guess .DCP format support could be done with something like this dcpTool http://dcptool.sourceforge.net/ https://sourceforge.net/projects/dcptool/ (crossplatform, free and open source command line tool)
Or at least dcpTool may help to extract and collect Base Curves for different cameras...
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302