Improvements and ideas for RAW workflow in PhotoLine

Hier diskutieren die Betatester von PhotoLine untereinander und mit den Entwicklern
Benutzeravatar
shijan
Mitglied
Beiträge: 1497
Registriert: Mo 23 Dez 2019 15:21
Wohnort: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Beitrag von shijan »

Seems PhotoLine can open DNG files with 3:1 and 5:1 compression. This is rather unique option. DNG compression like this used only in some Blackmagic Design cameras and most RAW editing apps can't read files like these at all.
DNG files from these cameras are very special. They designed specially for DaVinci Resolve editor, and so raw data encoded in camera to 12 bit Log instead of 14 or 16 bit Linear. Also seems these DNGs contain incomplete metadata (or it is just missed for purpose), so colors are represented as BMDFilm Wide Gamut and don't transformed correctly to working color space. These files require manual calibration and custom made input ICC profile or input LUT. Those compressed DNGs behave the same "incorrectly" in other apps as well.
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
arnold_s
Mitglied
Beiträge: 66
Registriert: So 04 Okt 2009 02:36

Re: Improvements and ideas for RAW workflow in PhotoLine

Beitrag von arnold_s »

shijan hat geschrieben: Sa 09 Jan 2021 20:31 I guess problems with highlights clipping also may be just because PhotoLine use too many points for curve. In other raw editing apps those camera curves are way more simplified.
For example here is how same curve look in darktable app and PhotoLine
BildBild
No answer up to now, so I try.
It is not the number of points, that matters for posterization. two horizontally-connected points preceeding a vertically connected point in a relevant distance building a stair in the curve could be sufficient to create posterization by decreasing color depth at least in an area. The same for highlight clipping (special cases: a stair on the right or the left border of the curve resuling in a peak at the beginning or the end of the result histogramm)
Also curve adjustment layers normally do no demosaicing (it's the process creating result colors from mixing colors of a bayer (or different, e.g. from x-trans-) sensor color filter.
Looking at your curve
Clipping: you could have meant an example not chown, the picture part chown at least reveals no clipping.
Regarding the curve, there ist no real highlight clipping, but the last part on the right side righthand from the last peek is not steep, so with low bitdepth, made stronger e.g. by gamma corrections, it could lead to visual clipping.
The same for the stair in the middle, which ist the first real difference compared to the darktable curve.
I would try removing the points in the stair and look, if the grey part recovers.
Normally you adjust curves in a way, that the curve is steeper in parts with peaks in a histogram, if that retains the natural look.
(That way contrast ist increased in parts, where you can see it, whether in parts with fewer pixels the decreased depth catches only a few eyes.)
The darktable curve respects the first big peak, whether the pl curve is - apart from the stair (repecting the last peak (?)) - nearly linear in the area containing relevant counts, which generates an effect similar to tonal correction without contrast weighting.
Strange choices, especially in combination.
Benutzeravatar
shijan
Mitglied
Beiträge: 1497
Registriert: Mo 23 Dez 2019 15:21
Wohnort: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Beitrag von shijan »

Yep, shifted curve points cause the problem. I also noticed that some cameras generate normal curve with less amount of points. Here is example from Fujifilm X-T4. Seems like this thing is still under early testing:
Bild
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
Benutzeravatar
shijan
Mitglied
Beiträge: 1497
Registriert: Mo 23 Dez 2019 15:21
Wohnort: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Beitrag von shijan »

Most modern RAW editors offer Anti Aliasing filtration. I guess in most cases those AA filters are just a Chroma Noise Reduction masked with High Pass filter, so i guess it is more-less possible to emulate it with current PL tools. But anyway, i guess special Anti Aliasing filter in ProtoLine filters list may be very useful.
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
Benutzeravatar
shijan
Mitglied
Beiträge: 1497
Registriert: Mo 23 Dez 2019 15:21
Wohnort: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Beitrag von shijan »

Here are some quick suggestions that may help to improve RAW workflow. Nothing special here. Currently all this could be done manually and recorded to single click action. I guess the only impossible yet task is to read and draw original Base Contrast Curve written in RAW file. I still have no idea if that curve actually provided inside RAW file by camera manufacturers or if it is some sort of custom-made Base Contrast Curve designed by developers of raw processing apps.
Base Contrast Curve also could be shaped manually in less precise way with combination of Luminance + Contrast (or Gamma + Contrast) sliders in Exposure tool.

So here are the ideas:

If ProPhotoRGB set in PhotoLine RAW File Preferences, ProPhotoRGB with Linear Gamma automatically should be set to debayered source raw image layer. (If possible i also would like to see ACES color space option in RAW File Preferences)
Transformation from Source Linear Gamma to Document Gamma Corrected ICC profile done with normal Color Management instead of current strange way with Gamma slider in Levels.
In this scenario any working ICC profile could be set to color managed Document or to color managed Group in "Draw Isolated" mode.
Also as you may know in PhotoLine "Document Mode" if switch Document to 32 bit, it is possible to do all processing in 32 bit, but keep original image Layer in 16 bit and so don't increase PLD file size.
In same way also possible to use "File Based Placeholder" workflow for RAW. In this scenario PLD will read external linked RAW file and so PLD size will be only few kilobytes in size.

Instead of original legacy ProPhoto g1.8 working ICC profile it could be introduced more modern variation of ProPhotoRGB primaries with sRGB or with L* TRC. Some modern raw editors also use ACES primaries with sRGB TRC as default working color space.

Used normal (non combined) Adjustment Layers tree. Everything placed inside Group. Group use name of the original RAW file. Easy to collapse into single compact layer.

Fixed Base Contrast Curve that don't auto changes randomly depending of image brightness and don't damage image look. I don't know yet if this curve should be used in Linear Gamma or in Gamma Corrected space by specification.

Instead of Levels slider, used "Exposure" slider. Easier to understand, easier to adjust exposure in f-stops. "Exposure" tool normally should be used in Linear Gamma, but i didn't test yet how Exposure tool depends of gamma in PhotoLine.

"Channel Mixer" renamed to easier to understand name "Input Color Matrix"

Bild
Zuletzt geändert von shijan am Di 14 Dez 2021 23:54, insgesamt 15-mal geändert.
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
Benutzeravatar
shijan
Mitglied
Beiträge: 1497
Registriert: Mo 23 Dez 2019 15:21
Wohnort: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Beitrag von shijan »

Another big question is how to adjust White Balance. It looks like PhotoLine can operate only with already debayered image, so real RAW WB adjustment is not available.
White Balance could be adjusted in Linear gamma in debayered images in same way as in RAW, but from my personal tests with other software large WB adjustments in non RAW files may generates some artifacts (see details here https://forum.blackmagicdesign.com/view ... 03#p631063) But all this may depend a lot of special tool design. Need to do the same tests in PhotoLine...

For Temperature/Tint adjustment tool with a little bit confused name "Color Correction" could be used. Adjust sliders manually or select grey point with Picker!
To pick auto WB could be also used "White Point" tool in "Set Grey Point" mode and with disabled "Fix White Point" or Channel Mixer with picker. There is also "Color Temperature" filter. It needs to be tested which one from these tools produce more correct WB adjustment.
UPD. "Color Correction" works somehow strange in Linear gamma. Not sure if this is a bug or some processing limitation in this filter.
"White Point" and "Color Temperature" are work as expected in Linear gamma.

Here are also some small improvement ideas for "Color Correction" filter UI:
1. Place Blue/Yellow (Temperature) at top and Green/Red (Tint) at bottom. This is the default arrangement of these sliders used in any existing image or video editing app.
2. Add user friendly "Temperature" and "Tint" hints to slider names.
Example sketch:
Bild
Zuletzt geändert von shijan am Do 16 Dez 2021 00:56, insgesamt 5-mal geändert.
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
Benutzeravatar
shijan
Mitglied
Beiträge: 1497
Registriert: Mo 23 Dez 2019 15:21
Wohnort: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Beitrag von shijan »

It is also very interesting to see that in some other raw editors Base Contrast Curve is not a fixed option, but organized as camera presets. So it could be changed by user to simulate different in-camera JPEG looks. https://docs.darktable.org/usermanual/3 ... ase-curve/
" base curve
Simulate the in-camera JPEG by applying a characteristic base curve to the image.
darktable comes with a number of base curve presets that attempt to mimic the curves of various camera manufacturers. These presets are automatically applied according to the manufacturer ID found in the image’s Exif data. Camera-specific base curve presets are also available for some camera models.
This module will be enabled by default if preferences > processing > auto-apply pixel workflow defaults is set to “display-referred”. A second option in the preferences dialog allows you to choose whether darktable should attempt to apply a camera-specific base curve (if found) or the generic manufacturer one."


I guess in PhotoLine this could be organized as curve presets as well. It should be some way to extract this curve data somehow...
Bild
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
Benutzeravatar
shijan
Mitglied
Beiträge: 1497
Registriert: Mo 23 Dez 2019 15:21
Wohnort: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Beitrag von shijan »

So for some further tests i decide to copy Base Curves from darktable using Hald cLUT pattern and convert them to 1D LUTs in .cube format. From subjective tests those curves where designed to work in Linear gamma and should be applied after Exposure adjustments (please confirm or deny this guess). Things works now same as in any normal RAW editor. I can Exposure in f-stops and see nice looking starting point without any additional manual guesses. Same time i can always turn off Base Curve adjustment layer and see original untouched "flat" source image look.
Note that it is not so easy to draw similar curve manually. Curves correction in Linear gamma needs high precision and even tiny wrong manual movement may affect image a lot. Curves like these usually generated based on calibration charts but not drawn manually by guess.
So this is how layers tree looks now:
Bild

And side by side compare to default PhotoLine RAW "auto curve" look (RAW example from Fujifilm X-T2 camera from https://www.dpreview.com/reviews/image-comparison)
Bild
Bild
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
Benutzeravatar
shijan
Mitglied
Beiträge: 1497
Registriert: Mo 23 Dez 2019 15:21
Wohnort: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Beitrag von shijan »

:arrow: So it is time to test White Balance tools in PhotoLine.
I exported two versions of same RAW image to 16 bit TIFF in ACES color space with Linear Gamma.
One image was White Balanced correctly in RAW editor before export. White Balance in another image version was adjusted in maximum possible wrong way. Note that due huge size of ACES color space color channels are not clipped after these extreme adjustments. This is important.
Base Curve and Noise Reduction where turned off in RAW editor to provide untouched "flat" TIFF output.
Bild
Bild

Image Layer set to ACES Color Space and Gamma 1.0 (Linear Gamma).
Document set to ACES Color Space with L* TRC.
Everything works in same way as in my RAW setup explained earlier:
Color Management transforms from 16 bit Linear Gamma Image Layer to 32 bit Gamma Corrected Document (ACES Color Space with L* TRC, or any other ICC profile that you prefer to use).
Adjustment Layers applied inside Image Layer operate in ACES Color Space and Gamma 1.0 (Linear Gamma Space)
Adjustment Layers applied on top of Image Layer operate in ACES Color Space with L* TRC (Gamma Corrected Space):
Bild

Manual Curve applied on top of everything just to make look correction results more visible:
Bild

Test results:
1. "Color Temperature" tool can work in both Linear Gamma Space and in Gamma Corrected space, but it don't have picker and "Tint" option, so it is useless in this situation.

2. "White Point" tool can work in both Linear Gamma Space and in Gamma Corrected space.
In "Setup Grey Point" mode and with disabled "Fix White Point" option in Linear gamma it produce perfect White Balance correction without any additional color casts, when picked grey color patch.
Bild

Bild

As expected in Gamma Corrected space correction is not so perfect:
Bild

Bild

Compare to WB adjusted directly in RAW editor i didn't notice any artifacts under deep magnification, but i noticed small difference in some colors. Both Iridient Developer and PhotoLine use the same LCMS color management system in Preferences. This difference in color could be because RAW editors adjust WB inside original camera input color space, before transformation to ACES or any other working color space.
Or this difference could be because DCP-based dual illuminant input profile in RAW editor adjusts color matrix coordinates depending of WB adjustment, but baked TIFF image use basic ICC-based input profile that can't react to WB adjustments in same way as DCP-based input profile:
Bild

3. "Channel Mixer" tool can work in both Linear Gamma Space and in Gamma Corrected space. It produce near the same result as "White Point" tool when picked grey color patch.
In Linear gamma it produce perfect White Balance correction without any additional color casts.
Bild

In Gamma Corrected space correction is not so perfect:
Bild

4. "Color Correction" tool seems like broken or simply don't designed for White Balance correction at all.
In Linear Gamma Space it became extremely sensitive, so tiny movements of the slider shift extremely huge amount of color
:arrow: UPD. Developers noticed me that this is really a bug. Problem fixed in 23.40b2, it works normal in Linear gamma now. But seems "Color Correction" is not designed for real WB adjustments and even in Linear Gamma it can't adjust image exactly back to original:
Bild

In Gamma Corrected space it also produce some color cast when picked grey color patch:
Bild

And updated layout chart:
Bild
Zuletzt geändert von shijan am Mi 29 Dez 2021 20:36, insgesamt 7-mal geändert.
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
Benutzeravatar
shijan
Mitglied
Beiträge: 1497
Registriert: Mo 23 Dez 2019 15:21
Wohnort: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Beitrag von shijan »

It is also appears that Base Curve concept is totally different between different apps:

ACR, Irident Developer and darktable apply Base Curve in Linear Gamma. That curve use fixed shape and matched to specific camera model, so you need manually adjust exposure to fit dynamic range of the image inside that curve limits. This concept needs additional manual adjustment but produce more stable and predictable results.

PhotoLine, as well as RawTherapee uses automatically generated curve (Auto Matched Tone Curve) applied in Gamma Corrected Space. This curve attempt to clip some empty highlights zone if exposure is too low. As a side effect, in some complicated images it may generate artifacts and unwanted clipping instead of proper auto correction. Also the tonal look of the image will change if you switch to different working color space with different gamma.
Bild
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
Benutzeravatar
shijan
Mitglied
Beiträge: 1497
Registriert: Mo 23 Dez 2019 15:21
Wohnort: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Beitrag von shijan »

Another important option for RAW module in PhotoLine could be support for .DCP input profiles. There are plenty of apps and tools that support this format. But currently in PhotoLine it is only possible set custom input profile for camera in ICC format.
Guess .DCP format support could be done with something like this dcpTool http://dcptool.sourceforge.net/ https://sourceforge.net/projects/dcptool/ (crossplatform, free and open source command line tool)
Or at least dcpTool may help to extract and collect Base Curves for different cameras...
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
Benutzeravatar
shijan
Mitglied
Beiträge: 1497
Registriert: Mo 23 Dez 2019 15:21
Wohnort: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Beitrag von shijan »

Here is another very simple and accurate way to develop RAW file in linear gamma and shape the contrast without clipped shadows/highlights (idea came from this thread viewtopic.php?f=7&t=7012)

Step by step workflow:
- Open RAW file and save Channel Mixer input matrix preset for your camera model.
Bild

- Download and install actions presets from attachment.
- Setup RAW Preferences like this:
Bild

- Open RAW photo. All default adjustments will be deleted automatically by action.
- Set ProPhotoRGB ICC profile with Linear gamma in Document Attributes.
- Add Correct Highlights adjustment to remove magenta artifact in highlights.
- Add Channel Mixer adjustment and use preset of earlier saved input matrix.
- Add Exposure adjustment.
- Optionally adjust Exposure to the point where is no any visible extreme highlights clipping.
- Increase Luminance.
- Decrease Gamma + increase Luminance if you want more contrast look.

Here is final look example and also preview of Gamma+Luminance curve (rendered using HaldCLUT in other app):
Bild
Dateianhänge
RAW as Placeholder Clear All.actions.zip
(400 Bytes) Noch nie heruntergeladen
RAW as Image Clear All.actions.zip
(382 Bytes) Noch nie heruntergeladen
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
Benutzeravatar
shijan
Mitglied
Beiträge: 1497
Registriert: Mo 23 Dez 2019 15:21
Wohnort: Ukraine

Re: Improvements and ideas for RAW workflow in PhotoLine

Beitrag von shijan »

Dual de-mosaic methods for RAW (DCB+VNG, Markesteijn+VNG)
UPD. Guess this question belongs to LibRaw developers...
PhotoLine UI Icons Customization Project: https://www.pl32.com/forum3/viewtopic.php?f=3&t=6302
Antworten