Color 'round Linux

As reported in my previous post Argyll software now offers in its latest 0.7 beta the support of the Spyder2. I've only recently found the time to do some testing.

First of all, I've downloaded the binary archive of the 0.7 branch from the Argyll website. Don't bother to compile everything by yourself, Argyll binaries are working "out of the box" in all the distros that I have (namely Gentoo and (K)Ubuntu). So it should work fine on yours.

Argyll will ask you to first build the driver support of your Spyder2 by using... the CD provided by Colorvision. Argyll will retrieve the .dll necessary to operate the Spyder2 from it and create a small driver package that dispcal will use to calibrate your screen.

While I had no problem with a relatively recent box from Colorvision, a CD from an older box didn't work and the Spyder2Pro drivers downloaded from the Colorvision website didn't work either. A quick fix here is to install the software under wine, and point spyd2en, the small utility from the Argyll suite, to the CVSpyder.dll somewhere in the Colorvision folder the install created.

From there, the calibration process is quite straight forward. I used:

./dispcal -yl -v2 -qm belinea

Where -y is the type of monitor (here l for lcd), -v2 (for a great verbosity) a -qm (for medium quality) and "belinea" is the name of the output file.

The result is good, and after going through some of my pictures, probably a bit better than the Colorvision utility. I still have to go through a "high" quality process to see if there is an improvement ("medium" was already way better than "low").

I've hear that the Argyll beta has been incorporated in Lprof which has a GUI, so I have to digg this as well. Stay tuned.

Argyll now integrates with the 0.7 beta version a way to support the Spyder2 from Colorvision. This is not a direct support though, as Argyll still needs to use the "genuine" binary compiled version delivered with any Spyder2 device. Through a dedicated Spyder2 module, Argyll will ask you to give him a path (CD or whatever) to the Colorvision/Spyder2 installation package. After that, you can perform a "regular" profiling of your monitor using the dedicated Argyll module.

I still have to explore this software and provide here some figures at how well it performs against a profile produced by the Colorvision software under the Windows OS.

After an introduction to a full workflow using Digikam for processing RAW pictures, it is time to have a closer look at the various advanced options the IO Files Settings has to offer. To open the IO Files Settings, go to the Configuration > Digikam Configuration > IO Files tab.

Highlights

Default is here to consider highlights (read: part of your pictures that are burned due to the inability of your camera to capture the highlights) as plain / solid white (solid white option). You can get some fancy results with the Unclip option which will paint the hightlights in various pinks. At last you can try to consider recovering some parts of the missing information from the hightlights (reconstruct option).

This is possible because the blue pixels tends to saturate less quickly than the greens and the reds. Digikam/Dcraw will try to reconstruct the missing green and red colors from the remaining none saturated blue pixels. Of course here everything is a question of tradeoff between how much color or white you want.

If you select reconstruct as the option, you will be given the choice to set a level. A value of 3 is a compromize and can/should be adapted on a per picture basis. Given my (limited) experience these past 3 months, 3 gives good results most of the time. A small warning here, for the few curious that have read the man pages of Dcraw, the author says that 5 is the compromize, 0 is solid white and 1 unclip. This is because in Digikam 0 and 1 are the "solid white" and "unclip" options in the drop down menu (if you select these, the level slider will be grayed out). Therefore, the slider in Digikam with the "reconstruct" option will let you choose between 0 to 7 (instead of 0 to 9 in Dcraw command line) where 3 is the compromise instead of 5 in "native" Dcraw command line tool.

Set the luminosity to your taste, you probably need to go a little higher than the default 1. But this can be adjusted later with the image options in the Digikam image editor. Given the cost in time for demosaicing, I would be relatively conservative here.

Quality

This is an interesting part, because we are going to set which algorithm will be used to demosaice our RAW pictures. Wikipedia has a very good article on what is demosaicing, briefly, as set in the introduction of the article:
A demosaicing algorithm is a digital image process used to interpolate a complete image from the partial raw data received from the color-filtered image sensor internal to many digital cameras in form of a matrix of colored pixels. Also known as CFA interpolation or color reconstruction.

Digikam and Dcraw offer us three alternatives: bilinear, VNG interpolation, AHD interpolation. Which one is the best? It seems that AHD interpolation (for Adaptive Homogeneity-Directed) is the best choice for quality according to some test that I have performed and the paper of the person that implemented it. VNG interpolation (Variable Number of Gradients) was the first algorithm used by Dcraw but suffers from color artifacts (to understand how color artifacts appears on the edge of a picture depending on the demosaicing algorithm you use, you can have a look at this explanation). Bilinear is interesting if you are looking for speed with a acceptable result (if you would like for instance to print your image in a small format, or for the web).

Interpolate RGB as four colors

If your pictures suffers from patterns (mazes, grids, etc.), if the automatic color balance didn't improve anything, you may try to set this option to on.

Enable Noise Reduction

While demosaicing your picture, you can also ask Digikam / Dcraw to perform a noise reduction. This option apply a noise reduction algorithm while your picture is in CIE Lab color space. Because the noise is only applied to the Luminosity layer (the "L" of the Lab), it is said to not blurr your picture as traditional noise reduction algorithm does in RVB mode.

Default options Domain (sigma_h or sigma_domain) set to 2 and Range (sigma_s or sigma_range) set to 4 are the default recommended options.

Digikam is a Photo/Picture Management software. It's aim is to help you organize your photo albums, manage tags... Since the recently release 0.9 (Xmas '06!), Digikam is also able to handle 16 bits pictures, raw format, and, thanks to lcms, color profiling.

This small howto will present what would be a color management workflow using Digikam starting from a raw camera file to a final png/jpg picture. I will not cover for now the ability Digikam offers to automate the process but rather focus on how to do it manually. But let's start...

New Digikam Options

Quite obviously you will need to have a running 0.9 Digikam (either unmask it, enable beta repositories, etc. depending on your distro) and install it. Done? Ok let's start it and begin with adjusting various new options Digikam now offers.

IO Files Settings

These options enable raw decoding by Digikam. For this Digikam embeds the dcraw engine. Dcraw is a powerfull (albeit slightly complex) tool for raw picture processing. Hopefully, Digikam hides most of this complexity:

Let's keep it basic for now, I will explore the various options in an other post, later:

• enable 16 bits color depth and the use camera white balance options.
• Set Highlights to reconstruct, with a level of "3" and a luminosity to 1. This has to do with picture saturation. As you might have notice if you have a picture with both deep area of shadows and a bright sky, the file generated by the camera either give priorities to the shadows and the sky is white (highlights are "burned") or the other way round and shadows are dark without any details. However, there is a solution to recover some of the highlights in the first situation (burned highlights). The blue pixels saturate less quickly than green and reds. From this information, Dcraw will try to recover some information by reconstructing the lost green and red information. This solution gives reasonably good results. Of course it is said to be less interesting than what the "fuji" sensor provides (if you have such a camera, you may be interested in turning on the Use Super-CCD secondary sensors).
• At last, set the Quality option to "Bilinear interpolation".

"Color Management"

Also new in Digikam 0.9 is the ability to color manage your pictures. Why so? Because raw files are not color managed at all. Your camera provides the data it has captured in a raw format and will let you manage all the processing. Each cameras have its specificities regarding how it captures color information, therefore you will need to apply a profile to the pictures you want to process, then translate those colors into an other profile with a wider gamut in order to be able to adjust the colors with more flexibility.

Basically, a profile "maps" the color information and gives information on how one should render them. It gives also information to lcms (and Digikam) on how to translate the color information from one space to an other in order to keep the colors as accurate as possible.

Here, the options you would like to set are the following:
Color Management Policy:

• Enable Color Management (obviously)
• Behaviour: Ask when open an image in image editor / Launch Color Management plugin with RAW files

Color Profiles Directory:

• set this to the Directory where you stores all your profiles... I'm using "/urs/share/color/icc" but it's not compulsory

ICC Profiles Settings: Here you are given the ability to provide "default" choices for your profiles. Everything is adaptable at the opening of a RAW file with the Color Management plugin...

• provide a workspace profile (such as Adobe RGB or sRGB). If you want to go to print with your pictures, you may want to go to Adobe RGB, if it is only for web publishing, sRGB is better (Adobe RGB will be displayed slightly dull in non color managed enable software such as browsers). However you may change this later of course, therefore Adobe RGB can be a good choice for storing and picture handling as you can always change it to sRGB before releasing a picture for your blog...
• input profile should match the camera maker you are using. I am the (happy) owner of a Minolta 7D and I therefore use a minolta profile.

There are some other options such as the soft proof profile which enables you to emulate, granted that you have a profile for it, how your picture will render for a particular device. It is useful before printing for instance because your printer has a smaller gamut than your camera and some colors might look saturated. You may want to fix this manually instead of relying on the "blind" algorithm of your printer.

The Use color managed view is an alternative to using Xcalib or Argyll. Only your picture will be color managed, not your entire screen! Despite the warning, this option should run smoothly in the final 0.9 release according to the Digikam team.

Process a raw picture

From there everything is simple, select a RAW picture and open it in the editor (double click, etc.). The color management plugin will be displayed after the picture has been processed in order to adjust color rendering. Without color management all pictures are very dark, this is normal.

You can experiment the various profiles you have in order to select the best input profile according to your picture (some tends to be very neutral, even a bit dull, some gives more saturated results, etc).

To do list

After this introductory post, I will explore in more details the Color Management and IO Files options (and their effects) provided by Digikam. Stay tuned.

We have loaded a profile created under Windows, is this profile accurate in Linux given that it is not processed the same way as under Windows?

My methodology:

• I've created the profile under MS Windows
• loaded under MS Windows and I've measured some color patches with the colorimeter (first line of the results)
• I've loaded the profile under Linux using Xcalib
• then I've measured the color patches through the same colorimeter but connected from a second computer (under Windows) with the same software version as used for my first measures. Results are on the second line here.
• Computed a DeltaE for each color measured to see the... delta.

The results:

white
39.944 / -32.287 / -14.397
40.152 / -32.366 / -14.561
DeltaE: 0.24

grey 90
% 34.923 / -28.010 / -13.397
35.495 / -28.177 / -13.498
DeltaE: 0.57

grey 60
% 22.958 / -18.877 / -8.160
23.678 / -19.231 / -9.119
DeltaE: 0.74

green
30.662 / -39.411 / 31.107
31.093 / -39.668 / 31.262
DeltaE: 0.43

blue
19.102 / -19.473 / -47.003
19.022 / -19.066 / -47.829
DeltaE: 1.51

red
14.134 / 18.997 / 8.038
13.879 / 20.905 / 6.802
DeltaE: 2.84

A DeltaE below 2, a human eye can (almost) not make the difference. Between 1-3, the results are excellent, and here all results are below 3. Given the very small differences involved, my colorimeter is proably not precise enough to be completely reliable from one measure to an other.

If this is of interest for someone, I am ready to post my methodology to provide you with the results above (in perticular computing the DeltaE).

There are a good and bad news. The good side is that you can run Linux and have a monitor calibrated as you would have in MS Windows. The bad side is that you must have a dual boot of some kind to calibrate your hardware under MS Windows and then you may load your profile under Linux using xcalib or Argyll.

Introduction

However if you use Argyll (dispcal utility) and are the happy owner of the Xrite DTP-94 and Xrite DTP-92 colorimeters that come in Monaco OPTIX and ColorEyes from the company Monaco/Xrite? you may have a 100% linux color management by calibrating your monitor under Linux as well.

Ok, I guess like most of you, I don't own one of these colorimeters, therefore I still need MS Windows. May be one better day... Consequence is that I've never calibrated a monitor (under Linux) and can't speak about that part. For the rest of this howto, I will take for granted that you managed to create a profile of some kind of your monitor, that you are now under Linux and would like to get your monitor corrected.

I will only explore how to load the profile of a single monitor. Two monitors might involves a little bit more work and is probably possible with certain limitation. One last word, on my knowledge as August 2006, loading a profile was not working with XGL/compiz or this kind of fancy 3D desktop. It might one day, but it's because the Xvidmode is not supported yet.

Installing Argyll and Xcalib

Under Linux, to my knowledge, there are two software that allows to load a profile in order to get one's monitor calibrated. These software are: Argyll and Xcalib. You may choose one of them as my experience shows that they behave the very same way and a profile loaded by one or an other looks exactly the same or seems to look exactly the same. However I encourage you to install both of them, as there are certain features present on the first and not the second and besides you will be in position to make up your mind about what's the best choice.

Let's start with Xcalib 0.6

Perequiste: XVidMode (XVidModeExtension enabled) -- the same apply for Argyll.

• Download Xcalib (see the links on the right side of this blog)
• Untar/bz2 or whatever, wherever you feel like
• Fire up a console and "Make" the software with the following command:

make lo_xcalib

Suprise there are several make you can do. Most notabily if you have an ATI, you may do instead:

make fglrx_xcalib

There are also the "make icclib_xcalib" and the "make lcms_xcalib", for detailed explanation, you may read the readme. However as we are in an exploratory phase the lo_xcalib provides a convenient no perequiste mode that contrast with the lcms version where you will have to patch the lcms library beforehand. I might investigate in future post if it makes sens to use the other options.

• If all went well as it should, you have now a nice xcalib program.
• do a

chmod +x xcalib

to be in order to execute it.

• We will leave it in its directory for now, let's build Argyll profile loader.

Second Argyll dispwin 0.6

• Download Linux binary Argyll suite of utilities (see the links on the right side of this blog)
• Untar/bz2 or whatever, wherever you feel like
• and here you are with the installation. In the bin folder you will find (among a LOT of other things) a working dispwin utility.
• (none compulsory) a patch has been made for people using 64bit OS and also people working with several screens and proprietary graphic (ATI / NVIDIA) drivers.

Playing with calibration

One advantage of Xcalib over Dispwin is that it offer the way to "clear" a profile from the X server and recover a none calibrated monitor (very usefull for testing the before/after with/without profile).

For Xcalib the main commands are (from within the xcalib directory or it won't work):

./xcalib /usr/share/color/icc/060731_belinea.icm
where /usr/share/color/icc is your path to your color profile. I would like to encourage you at using the above location. Seems to become the default. But, do as you wish if you would like to do otherwise.

To see if your profile is working, you may also use the supplied "bluish" profile.

./xcalib bluish.icc

Pleased with the result? Yes, so let's clear out the result and see how Argyll works.

./xcalib -clear
to clear out all profile of the X server

Go into your Argyll directory and in the bin folder. With the very same console as above simply type:

./dispwin

and you will see the following color patches: Red, Green, Blue, Cyan, Magenta, Yellow and then it will darken your screen, lighten it and restore the previous parameters. Well done, dispwin is working fine. Issue now the following command:

./dispwin /usr/share/color/icc/060731_belinea.icm

It will load your profile. You may also use the bluish profile of Xcalib, it will work just as fine. You may play between dispwin and xcalib with commands like this:

xcalib -clear && xcalib /usr/share/color/icc/060731_belinea.icm

and try to see if there is any difference.

Finishing the installation

First install the utility of your choice or both where you like. You can even keep it where you install it, but only your user will be able to use it. Otherwise, you may install it under /usr/bin or /usr/local/bin as most of the time these two locations are already on your PATH (ie accessible from your console without any fuss).

Under root do as this:

cp xcalib /usr/bin/

Same thing applies for dispwin.

To automate the process of loading a profile at startup, if you are using KDE (but the very same thing exist for GNOME), you can write a script that you will put under for exemple ~/.kde3.5/Autostart/ where kde3.5, 3.5 is your version of KDE. In case you don't know "~" is your home folder. In my case "/home/nicolas/". Your script will look at something like this (I am using xcalib, change it to dispwin if you prefer dispwin):

#!/bin/bash
xcalib /usr/share/color/icc/060731_belinea.icm &

Don't forget to change it to be executable

chmod +x your_script

In my case, "your_script" has been named "calibration2". For the sake of it, you might want to log off and log in to see if it works. Et voilà, you have a working calibrated environment, let's start the real work on pictures now!

I would like first to give some general usefull links about Linux Color Management.

• THE color management Linux link. The articles provides lots of good information and points to several usefull information site.
• About xcalib and Argyll, kind of article where the solutions lies in the post ''below'' the article. But nice because everything is in there. I will try myself to put more screenshot and some clarity on how to procede exactly step by step to get some good results.
• Yet an other article-and-post kind

the main purpose of this site will to present several technics around color management using Linux. I will also time to time write some basic howtos on managing pictures using Open Source tools.

As today, I am using the following tools :

• KDE 3.5.5 as desktop
• Digikam as photo management tool
• xcalib to load my display color profile