Advanced Color Image Processing and Analysis

Christine Fernandez-Maloigne
Editor

Advanced Color Image

Processing and Analysis

123
Editor
Christine Fernandez-Maloigne
Xlim-SIC Laboratory
University of Poitiers
11 Bd Marie et Pierre Curie
Futuroscope
France

ISBN 978-1-4419-6189-1 ISBN 978-1-4419-6190-7 (eBook)

DOI 10.1007/978-1-4419-6190-7
Springer New York Heidelberg Dordrecht London
Library of Congress Control Number: 2012939723

© Springer Science+Business Media New York 2013

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection
with reviews or scholarly analysis or material supplied specifically for the purpose of being entered
and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of
this publication or parts thereof is permitted only under the provisions of the Copyright Law of the
Publisher’s location, in its current version, and permission for use must always be obtained from Springer.
Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations
are liable to prosecution under the respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for
any errors or omissions that may be made. The publisher makes no warranty, express or implied, with
respect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media ([Link])

Preface

Color is life and life is color!

We live our life in colors and the nature that surrounds us offers them all, in all
their nuances, including the colors of the rainbow. Colors inspire us to express our
feelings. We can be “red in the face” or “purple with rage.” We can feel “blue with
cold” in winter or “green with envy,” looking at our neighbors’ new car. Or, are we
perhaps the black sheep of our family? ....
Color has accompanied us through the mists of time. The history of colors is
indissociable, on the cultural as well as the economic level, from the discovery of
new pigments and new dyes. From four or five at the dawn of humanity, the number
of dyes has increased to a few thousands today.
Aristotle ascribed color and light to Antiquity. At the time, there was another no-
tion of the constitution of colors: perhaps influenced by the importance of luminosity
in the Mediterranean countries, clearness and darkness were dominating concepts
compared to hues. Elsewhere, colors were only classified by their luminosity as
white and black. Hues were largely secondary and their role little exploited. It should
be said that it was rather difficult at that time to obtain dyes offering saturated colors.
During the Middle Ages, the prevalence of the perception of luminosity continued to
influence the comprehension of color, and this generally became more complicated
with the theological connotations and with the dual nature of light declining in
Lumen, the source of light of divine origin (for example, solar light) and Lux, which
acquires a more sensory and perceptual aspect like the light of a very close wood
fire, which one can handle. This duality is included in the modern photometric units
where lumen is the unit that describes the flow of the source of light and Lux is the
unit of illumination received by a material surface. This design based on clearness,
the notion taken up by the painters of the Renaissance as well under the term of
value, continues to play a major role, in particular for graphic designers who are
very attached to the concept of the contrast of luminosity for the harmony of colors.
In this philosophy, there are only two primary colors, white and black, and the other
colors can only be quite precise mixtures of white and black. We can now measure
the distance that separates our perception from that of the olden times.

v
vi Preface

Each color carries its own signature, its own vibration. . . its own universal
language built over millennia! The Egyptians of Antiquity gave to the principal
colors a symbolic value system resulting from the perception they had of natural
phenomena in correlation with these colors: the yellow of the sun, the green of
the vegetation, the black of the fertile ground, the blue of the sky, and the red of
the desert. For religious paintings, the priests generally authorized only a limited
number of colors: white, black, the three basic colors (red, yellow and blue), or their
combinations (green, brown, pink and gray). Ever since, the language of color has
made its way through time, and today therapeutic techniques use colors to convey
this universal language to the unconscious, to open doors to facilitate the cure.
In the scientific world, although the fundamental laws of physics were discovered
in the 1930s, colorimetrics had to await the rise of data processing to be able to use
the many matrix algebra applications that it implies.
In the numerical world, color is of vital importance, as it is necessary to code and
to model, while respecting the basic phenomena of the perception of its appearance,
as we recall in Chaps. 1 and 2. Then color is measured numerically (Chap. 3),
moves from one peripheral to another (Chap. 4), is handled (Chaps. 5–7), to
extract automatically discriminating information from the images and the videos
(Chaps. 8–11) to allow an automatic analysis. It is also necessary to specifically
protect this information, as we show in Chap. 12, to evaluate its quality, with
the metrics and standardized protocols described in Chap. 13. It is with the two
applications in which color is central, the field of art and the field of medicine, that
we conclude this work (Chaps. 14 and 15), which has brought together authors from
all the continents.
Whether looked at as a symbol of joy or of sorrow, single or combined, color is
indeed a symbol of union! Thanks to it, I met many impassioned researchers from
around the world who became my friends, who are like the members of a big family,
rich in colors of skin, hair, eyes, landscapes, and emotions. Each chapter of this will
deliver to you a part of the enigma of digital color imaging and, within filigree, the
stories of all these rainbow meetings. Good reading!
Contents

1 Fundamentals of Color.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1
M. James Shyu and Jussi Parkkinen
2 CIECAM02 and Its Recent Developments .. . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 19
Ming Ronnier Luo and Changjun Li
3 Colour Difference Evaluation . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 59
Manuel Melgosa, Alain Trémeau, and Guihua Cui
4 Cross-Media Color Reproduction and Display Characterization .. . . . 81
Jean-Baptiste Thomas, Jon Y. Hardeberg, and Alain Trémeau
5 Dihedral Color Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 119
Reiner Lenz, Vasileios Zografos, and Martin Solli
6 Color Representation and Processes with Clifford Algebra . . . . . . . . . . . 147
Philippe Carré and Michel Berthier
7 Image Super-Resolution, a State-of-the-Art Review
and Evaluation .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 181
Aldo Maalouf and Mohamed-Chaker Larabi
8 Color Image Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 219
Mihai Ivanovici, Noël Richard, and Dietrich Paulus
9 Parametric Stochastic Modeling for Color Image
Segmentation and Texture Characterization . . . . . . . .. . . . . . . . . . . . . . . . . . . . 279
Imtnan-Ul-Haque Qazi, Olivier Alata, and Zoltan Kato
10 Color Invariants for Object Recognition .. . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 327
Damien Muselet and Brian Funt
11 Motion Estimation in Colour Image Sequences . . . . .. . . . . . . . . . . . . . . . . . . . 377
Jenny Benois-Pineau, Brian C. Lovell, and Robert J. Andrews

vii
viii Contents

12 Protection of Colour Images by Selective Encryption .. . . . . . . . . . . . . . . . . 397

W. Puech, A.G. Bors, and J.M. Rodrigues
13 Quality Assessment of Still Images . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 423
Mohamed-Chaker Larabi, Christophe Charrier,
and Abdelhakim Saadane
14 Image Spectrometers, Color High Fidelity, and Fine-Art
Paintings .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 449
Alejandro Ribés
15 Application of Spectral Imaging to Electronic Endoscopes . . . . . . . . . . . . 485
Yoichi Miyake

Index . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 499
Chapter 1
Fundamentals of Color

M. James Shyu and Jussi Parkkinen

The color is the glory of the light

Jean Guitton

Abstract Color is an important feature in visual information reaching the human

eye or an artificial visual system. The color information is based on the electromag-
netic (EM) radiation reflected, transmitted, or irradiated by an object to be observed.
Distribution of this radiation intensity is represented as a wavelength spectrum. In
the standard approach, color is seen as human sensation to this spectrum on the
wavelength range 380–780 nm. A more general approach is to manage color as color
information carried by the EM radiation. This modern approach is not restricted to
the limitations of human vision. The color can be managed, not only in a traditional
three-dimensional space like RGB or L∗ a∗ b∗ but also in an n-dimensional spectral
space. In this chapter, we describe the basis for both approaches and discuss some
fundamental questions in color science.

Keywords Color fundamentals • Color theory • History of color theory • Col-

orimetry • Advanced colorimetry • Electromagnetic radiation • Reflectance spec-
trum • Metamerism • Standard observer • Color representation • Color space •
Spectral color space • n-dimensional spectral space • Color signal • Human
vision • Color detection system

M.J. Shyu ()

Department of Information Communications, Chinese Culture University, Taipei, Taiwan
e-mail: mjshyu@[Link]
J. Parkkinen
School of Computing, University of Eastern Finland, Joensuu, Finland
School of Engineering, Monash University Sunway Campus, Selangor, Malaysia
e-mail: jussi@[Link]

C. Fernandez-Maloigne (ed.), Advanced Color Image Processing and Analysis, 1

DOI 10.1007/978-1-4419-6190-7 1,
© Springer Science+Business Media New York 2013
2 M.J. Shyu and J. Parkkinen

1.1 Everything Starts with Light

The ability of human beings to perceive color is fantastic. Not only does it make
it possible for us to see the world in a more vibrant way, but it also creates the
wonder that we can express our emotions by using various colors. In Fig. 1.1, the
colors on the wooden window are painted with the meaning of bringing prosperity.
In a way, we see the wonderful world through the colors as a window. There are
endless ways to use, to interpret, and even to process color with the versatility that
is in the nature of color. However, to better handle the vocabulary of color, we need
to understand its attributes first. How to process as well as analyze color images
for specific purposes under various conditions is another important subject which
further extends the wonder of color.
In the communication between humans, color is a fundamental property of
objects. We learn different colors in our early childhood and this seems to be obvious
for us. However, when we start to analyze color more accurately and, for example,
want to measure color accurately, it is not so obvious anymore. For accurate color
measurement, understanding, and management, we need to answer the question:
What is color?

Fig. 1.1 A colorful window

with the theme of bringing
prosperity (Photographed by
M. James Shyu in Pingtong,
Taiwan)
1 Fundamentals of Color 3

In a common use of the term and as an attribute of an object, color is treated

in many ways in human communication. Color has importance in many different
disciplines and there are a number of views to the color: in biology, color vision and
colorization of plants and animals; in psychology, color vision; in medicine, eye
diseases and human vision; in art, color as an emotional experience; in physics, the
signal carrying the color information and light matter interaction; in chemistry,
the molecular structure and causes of color; in technology, different color measuring
and display systems; in cultural, studies color naming; and in philosophy, color as
an abstract entity related to objects through language [2, 9, 28].
It is said that there is no color in the light—to quote Sir Isaac Newton, “For the
Rays to speak properly are not coloured. In them there is nothing else than a certain
Power and Disposition to stir up a Sensation of this or that Colour” [21, 26]. It is
the perception of human vision that generates the feeling of color. It is the perceived
color feeling of the human vision defining how we receive the physical property
of light. Nevertheless, if color is only defined by human vision, it leaves all other
animals “color blind.” However, it is known that many animals see colors and have
an even richer color world than human being [13, 19].
The new technological development in illumination and in camera and display
technology requires new way of managing colors. RGB or other three-dimensional
color representations are not enough anymore. The light-emitting diodes (LED) are
coming into illumination and displays rapidly. There, the color radiation spectrum is
so peaky that managing it requires a more accurate color representation than RGB.
There exist also digital cameras and displays, where colors are represented by four
or six colors. Also this technology requires new ways to express and compute color
values.
Therefore, if we want to understand color thoroughly and be able to manage color
in all purposes, where it is used today, we cannot restrict ourselves to the human
vision. We have to look color through the signal, which causes color sensation by
humans. This signal we call color signal or color spectrum.

1.2 Development of Color Theory

In color vocabulary, black and white are the first words to be used as color names
[2]. After them when the language develops, come red and yellow. The vocabulary
is naturally related to the understanding of nature. Therefore in ancient times, the
color names were related to the four basic elements of the world, water, air, fire,
and earth [9]. In ancient times, the color theory was developed by philosophers like
Plato and Aristotle. For the later development of color theory, it is notable that white
was seen as a basic color. Also the color mixtures were taken into theories, but each
basic color was considered to be a single and separate entity [14].
Also from the point of view of the revolution of color theory by Newton [20], it is
interesting to note that Aristotle had a seven basic color scale, where colors crimson,
4 M.J. Shyu and J. Parkkinen

a
1
0.9
0.8
B
0.7
G
0.6
R
0.5
Y
0.4
M
0.3
C
0.2
0.1
0
380 430 480 530 580 630 680 730

Fig. 1.2 (a) A set of color spectra (x-axis: wavelength from 380 to 730 nm, y-axis: reflectance
factor) and (b) the corresponding colors

violet, leek-green, deep blue, and gray or yellow formed the color scale from black
to white [9]. Aristotle also explains the color sensation so, that color sets the air in
movement and that movement extends from object to the eye [24].
From these theories, one can see that already in ancient times, there exists the
idea of some colors to be mixtures of primary colors and seven primary colors.
Also, it is easy to understand the upcoming problems of Newton’s description of
colors, when the view was that each primary color is a single entity and the color
sensation was seen as a kind of mechanical contact between light and the eye. The
ancient way of thinking was strong until the Seventeenth century.
In the middle of the Seventeenth century, the collected information was enough
to break the theory of ancient Greek about light and color. There were a number
of experiments by prism and color in the early Seventeenth century. The credit
for the discovery of the nature of light as a spectrum of wavelengths is given to
Isaac Newton [20]. The idea that colors are formed as a combination of different
component rays, which are immaterial by nature, was revolutionary at Newton’s
time. It broke the strong influence of ancient Greek thinking. This revolutionary
idea was not easily accepted. A notable person was Johann Wolfgang von Goethe,
who was still in the Nineteenth century opposing Newton’s theory strongly [10].
Newton also presented colors in a color circle. In his idea, there were seven
basic colors: violet, indigo, blue, green, yellow, orange, and red [14]. In the spectral
approach to color as shown in Fig. 1.2, the wavelength scale is linear and continuing
1 Fundamentals of Color 5

both ends, UV from short wavelengths and IR from long wavelengths. From the first
look, the circle form is not natural for this physical signal. However, when the hu-
man perception of the different wavebands is considered, the circle form seems to be
a good way to represent colors. The element, which connects the both ends of visible
spectrum into a circle, is purple, which includes both red and violet part of spectrum.
The first to present colors in a circle form was the Finnish mathematician and
astronomer Sigfrid Forsius in 1611 [14]. There are two different circle representa-
tion and both are based on idea to move from black to white through different color
steps. Since Forsius and Newton, there are a number of presentations of colors on a
circle. The circular form is used for the small number of basic colors. For continuous
color tones, three-dimensional color coordinate systems form other shapes like cone
(HSV) and cube (RGB).
Next important phase in the development of color science was the Nineteenth
century. At that time, theories of human color vision were developed. In 1801,
the English physicist and physician Thomas Young restated an earlier hypothesis
by the English glassmaker George Palmer from the year 1777 [14]. According to
these ideas, there are three different types of color-sensitive cells in the human
retina. In their model, these cells are sensitive to red, green, and violet and to
other colors which are mixtures of these principal pure colors. German physicist
Hermann Helmholz studied this model further. He also provided the first estimates
of spectral sensitivity curves for the retinal cells. This is known as the Young—
Helmholz theory of color vision.
In the mid-Nineteenth century the Young—Helmholz theory was not fully
accepted and in the mid-1870s German physician and physiologist Karl Hering
presented his theory of human color vision [14]. His theory was based on four fun-
damental colors: red, yellow, green, and blue. This idea is the basis for the opponent
color theory, where red—green and blue—yellow form opponent color pairs.
Both theories, the Young—Helmholz theory of color vision and the Hering
opponent color theory, seemed to give a valid explanation to many observations
about the human color vision. However, they were different even in the number
of principal or fundamental colors. German physiologist Johannes von Kries
proposed a solution to this confusion. He explained that the Young—Helmholz
theory explained color vision on retinal color-sensitive cells level, and the Hering’s
opponent color theory was explaining color processes later in visual pathway [14].
This description was not accepted for some years, but currently it is seen as the basic
view about the human color vision.
These ideas were bases for human color vision models, for the trichromatic color
theories, and the standards of representing colors on a three-dimensional space.
However, basing color representation and management on trichromatic theory of
human color vision is very restrictive in many ways. Standard three-dimensional
color coordinates are useful in many practical settings, where color is managed
for humans to look at, especially under fixed illumination. However, there are also
several drawbacks in the color representation based on human color vision.
Current level of measurement accuracy has led to a situation where in
the equations for calculating color coordinates or color differences have become
6 M.J. Shyu and J. Parkkinen

complicated. There are number of parameters, many without explaining the theory,
but fitting the measurements to correspond the model. Furthermore, there are
a number of issues, which cannot be managed by trichromatic color models.
These include, e.g., fluorescence, metamerism, animal color vision, and transfer of
accurate color information. To overcome these drawbacks, spectral color science
has increased interest and is used more and more in color science.
As mentioned above, the basis of color is light, a physical signal of electromag-
netic radiation. This radiation is detected by some detection system. If the system is
human vision, then we consider traditional color. If we do not restrict the detection
system, we consider the physical signal, color spectrum. Kuehni separates these
approaches into color and spectral color.

1.3 Physical Attributes of Color

The color of an object can be defined as an object’s physical attribute or as an

object’s attribute as humans see it. The first one is measurable attribute, but what
humans see we cannot measure, since it happens in the human brain. In both
definitions, the color information is carried to the color detector in the form of
electromagnetic radiation. If the detector is human eye, seeing color of an object
is based on how human eye senses the electromagnetic signal reaching the eye and
how this sensory information is forwarded to the brain. In the artificial color vision
systems, the signal reaches the detector and the detector response is related to the
wavelength sensitivity of detector. This detected sensitivity information can then be
managed the way the system requires.
The detector response Di of ith detector to the color signal l(λ )r(λ ) is given as

Di = l(λ ) r(λ )si (λ )d λ , i = 1, n (1.1)

where l(λ ) is the spectrum of illumination, r(λ ) is the reflectance spectrum of the
object, si (λ ) is the sensitivity of the ith detector, and n is the number of detectors.
If the detector system has only one detector, it sees only intensity differences
and not colors. Or we can also say that the detector sees only intensities of one
color, i.e., color corresponding to the sensitivity s(λ ). For color sensation, at least
two detectors w ith different wavelength sensitivities are needed (n ≥ 2). The ratio
of these different detector responses gives the color information. In the human eye,
there are three types of wavelength-sensitive cone-cells (n = 3). These cells collect
the color information from the incoming signal and human visual system converts
it into color we see.
When we consider the color of an object, an essential part of color detection is
the illumination. Since the color signal is originally light reflected (or radiated, or
transmitted) from an object, the color of the illumination also affects to the detected
object’s color, term l(λ )r(λ ) in (1.1). A schematic drawing of detection of object
color is shown in Fig. 1.3.
1 Fundamentals of Color 7

Fig. 1.3 The light source,

color objects, and human
visual system are needed to
generate the perception of
color

Here we have the approach that the color information is carried by the electro-
magnetic signal coming from the object and reaching the detector system. For this
approach, we can set to the color signal certain assumptions.
Reflectance spectrum r(λ ) (or color spectrum l(λ )r(λ )) can be represented as a
function r: Λ → R, which satisfies

(a) r(λ ) is continuous on Λ

(b) r(λ ) ≥ 0λ ∈ Λ (1.2)
(c) ∫ |r(λ )|2 d λ < ∞

The proposition can be set due to the physical properties of the electromagnetic
radiation. It means that reflectance (radiance or transmittance) spectra and color
spectra can be thought as members of the square integrable function space,
L2 . Since in practice the spectrum is formed as discrete measurements of the
continuous signal, the spectra are represented as vectors in the space Rn . If spectra
are represented in a low-dimensional space, they lose information, which causes
problems like metamerism.
Using the vector space approach to the color, there are some questions to consider
related to the color representation:
– What are the methods to manage color accurately?
– What is the actual dimensionality of color information?
– How to select the dimensions to represent color properly?
In the case of standard color coordinates, the dimensionality has been selected to be
three. This is based on the models of the human color vision. Models are based on
the assumption that there are three types of color sensitivity functions in the human
retina.
In the spectral approach, originally the color signal is treated by using linear
models [17, 18, 22, 23, 25]. The most popularly used and the standard method is the
principal component analysis (PCA).
In this view, colors are represented as inner products between color spectrum and
basis spectra of defined coordinate system. This approach unifies the ground of the
different methods of the color representation and analysis. The basis spectra can be
defined, e.g., by human response curves of three colors or by the interesting colors
using some learning algorithm, depending on the needs and applications.
8 M.J. Shyu and J. Parkkinen

The use of inner products means seeing low-dimensional color representation

as projection of original color signal onto a lower dimensional space. This leads
to many theoretical approaches in the estimation of accurate color signal from
the lower dimensional representation, like RGB. It is not possible to reconstruct
the original color spectrum, from the RGB values of an object color. In theory, there
is an infinite number of spectra, which produce the same RGB-value for spectra
under fixed illumination conditions. However, if the original color spectra are from
the certain limited region in the n-dimensional spectral space, a rather accurate
reconstruction is possible to reach.
The considering of spectral color space as an n-dimensional vector space gives
a basis for more general color theory to form. In this approach, human color vision
and models based on that would be special cases. Theoretical frameworks, which
have been studied as the basis for spectral color space, include, e.g., reproducing
kernel Hilbert space [11, 23] and cylindrical spaces [16].

1.4 Standard Color Representation

In the case of human eye n = 3 in (1.1) and si (λ )’s are marked as x̄(λ ), ȳ(λ ), z̄(λ )
and called color matching functions [27]. This leads to the tristimulus values X, Y,
and Z

X=k l(λ ) r(λ )x̄(λ ) d λ

Y=k l(λ ) r(λ )ȳ(λ ) d λ (1.3)

Z=k l(λ ) r(λ )z̄(λ ) d λ

k = 100/ l(λ )ȳ(λ ) d λ

Moreover, three elements are involved for a human to perceive color on an object:
light source, object, and observer. The physical property of the light source and
the surface property of the object can be easily measured in their spectral power
distribution with optical instruments. However, the observer’s sensation of color
cannot be measured directly by instruments since there is no place to gather a direct
reading of perception. Equation (1.3) represents an implicit way to describe the
human color perception in a numerical way which makes it possible to bring the
human color perception into a quantitative form and to further compute or process it.
This implicit model to describe human color perception can be observed by the
color-matching phenomena of two physically (spectrally) different objects which
appear as the same color to the human eye, in the following equations:
1 Fundamentals of Color 9

2.5
x2
y2
2 z2
x10
1.5 y10
z10

0.5

0
380 420 460 500 540 580 620 660 700 740 780

Fig. 1.4 Color matching functions for CIE standard observer in 2 and 10◦ -degree viewing angles

 
l(λ ) r1 (λ ) x̄(λ ) d λ = l(λ ) r2 (λ ) x̄(λ ) d λ
 
l(λ ) r1 (λ ) ȳ(λ ) d λ = l(λ ) r2 (λ ) ȳ(λ ) d λ (1.4)
 
l(λ ) r1 (λ ) z̄(λ ) d λ = l(λ ) r2 (λ ) z̄(λ ) d λ

Due to the integral operation in the equations, there can be two sets of different
spectral reflectance of two objects that cause the equality to happen, i.e., make
them appear as the same color. Furthermore, with the known (measurable) physical
stimuli in the equations, if the unknown color-matching functions (x̄(λ ), ȳ(λ ), z̄(λ ))
can be derived for the human visual system, it is possible to predict whether two
objects of different spectral power distribution would appear as equal under this
human visual color-matching model.
It was the Commission International de l’Eclairage (CIE) that in 1924 took the
initiative to set up a Colorimetry Study Committee to coordinate the derivation of the
color-matching functions [6]. Based on experimental color-mixture data and not on
any particular theory of the color vision process, a set of color-matching functions
for use in technical Colorimetry was first presented to the Colorimetry Committee
at the 1931 CIE sessions [6]. This “1931 Standard Observer” as it was then called
was based on observations made with colorimeters using field sizes subtending
2 degrees. In 1964, the CIE took a further step to standardizing a second set of
color-matching functions as the “1964 Standard Observer” which used field sizes
subtending 10 degrees. With these two sets of color-matching functions, shown in
Fig. 1.4, it is possible to compute human color perception and subsequently open
up promising research in the world of color science based on the model of human
vision.
10 M.J. Shyu and J. Parkkinen

1.5 Metamerism

A property of color, which gives understanding about differences between human

and spectral color vision approaches, is metamerism. Metamerism is a property,
where two objects, which have different reflectance spectra, look the same under
a certain illumination. As sensor responses, this is described in the form of (1.4).
When the illumination changes, the object colors may look different.
The metamerism is a problem, e.g., in textile industry and in paper industry, if not
taken care of. In paper industry, when a colored newspaper is used, a newspaper may
be printed on papers produced on different days. If the required color is defined, e.g.,
by CIELAB coordinates and in the quality control only those values are monitored,
color of different pages may look different under certain illumination although the
pages appeared to have the same color under control illumination.
The metamerism is also used as a benefit. The most accurate way to reproduce the
color of an object on the computer or TV screen would be the exact reconstruction of
the original spectrum. This is not possible due to the limited number and shapes of
the spectra of display primary colors. Therefore, a metameric spectrum of the
original object is produced on the display and the object color looks to the human
eye the same as the original color.
In the literature, metamerism is discussed mainly for the human visual system,
but it can be generalized to any detection system with sufficient small number of
detectors (Fig. 1.5). This means that
 
1(λ ) r1 (λ ) si (λ ) dλ = 1(λ ) r2 (λ ) si (λ ) dλ for all i (1.5)

Fig. 1.5 Example of metamerism: two different reflectance curves from a metameric pair that
could appear as the same color under specific illumination
1 Fundamentals of Color 11

Another aspect related to the color appearance under two different conditions
is the color constancy. It is a phenomenon, where the observer considers the object
color the same under different illuminations [3]. It means that the color is understood
to be the same although the color signal reaching the eye is different under different
illuminations. The color constancy can be seen as related to a color-naming problem
[8]. The color constancy is considered in the Retinex theory, which is the basis, e.g.,
for the illumination change normalized color image analysis method [15].
In the color constancy, the background and the context, where the object is seen,
are important for constant color appearance. If we look at a red paper under white
illumination on a black background, it looks the same as that of a white paper
under red illumination on a black background [8]. This implicit model to describe
human color perception can be observed by the color-matching phenomena of two
physically different objects that appear to be the same color to human eyes.

1.6 Measuring Physical Property or Perceptual

Attribute of Color

The measurement of color can be done in various ways. In printing and publishing,
the reflection densitometer has been used historically in prepress and pressroom
operations for color quality control. ISO standard 5/3 for Density Measurement—
Spectral Conditions defines a set of weightings indicating the standard spectral
response for Status A, Status M, and Status T filters [1]. Reflectance density (DR ) is
calculated from spectral reflectance according to the following equation:

DR = − log10 [Σr(λ )Π (λ ) /ΣΠ(λ )] (1.6)

where
r(λ ) is the reflectance value at wavelength λ of the object measured
Π(λ ) is the spectral product at wavelength λ for the appropriate density response
It is well known that densitometers can be used to evaluate print characteristics
such as consistency of color from sheet to sheet, color uniformity across the sheet,
and color matching of the proof. According to (1.5), one can find that for two prints
of the same ink, if the reflectance values r(λ ) are the same, it is certain that the
density measures will be the same, i.e., the color of the prints will appear to be the
same. However, it is also known that for two inks whose narrow-band density values
have been measured as identical could appear as different colors to the human eye
if their spectral characteristics are different in the insensitive dead zone of the filter
[5]. It must be pointed out that due to the spectral product at each wavelength, prints
even with the same density values but not with the same ink have not necessarily
the same spectral reflectance values, i.e., they can appear as different colors to the
human eye. Since the spectral product in densitometry is not directly related to
12 M.J. Shyu and J. Parkkinen

Fig. 1.6 (a) Gray patches with the same color setting appear as the same color. (b) The same
patches in the center appear as different levels of gray due to the “simultaneous contrast effect”
where the background influence makes the central color patches appear different

human visual response, the density measure can only guarantee the equality of the
physical property of the same material, not the perceptual attribute of the color that
appears.
There are similarities and differences between Densitometry and Colorimetry.
Both involve integration with certain spectral weightings, but only the spectral
weighting in the color-matching functions in Colorimetry is directly linked to the
responsivity of human color vision. The measurement of color in the colorimetric
way defined in (1.3) is therefore precisely related to the perceptual attribute of
human color vision.
On the other hand, the resulting values of Colorimetry are more into the percep-
tual measurements of human color response. By definition in (1.4), if the spectral
reflectance of r1 (λ ) and r2 (λ ) are exactly the same, this “spectral matching” method
can of course create the sensation of two objects of the same color. However, it is
not necessary to constrain the reflectance of the two objects to be exactly the same,
as long as the integration results are the same, the sensation of color equality would
occur, which is referred as “colorimetric matching.” These two types of matching
are based on the same physical properties of the light source and the same adaptation
status of the visual system, which is usually referred as “fundamental Colorimetry”
(or simple CIE XYZ tristimulus system).
Advanced Colorimetry usually refers to the color processing that goes beyond
the matching between simple solid color patches or pixels, where spatial influence,
various light sources, different luminance levels, different visual adaptation, and
various appearance phenomena are involved in a cross media environment. These
are the areas on which active research into Color Imaging focuses and the topics
covered in the subsequent chapters. One example is shown in Fig. 1.6a where all
the gray patches are painted with the same R, G, and B numbers and appear
as the same color in such circumstances. However, the same gray patches with
different background color patches now appear as different levels of gray as shown
in Fig. 1.6b. This so-called “simultaneous contrast effect” gives a good example of
how “advanced Colorimetry” has to deal with subjects beyond the matching among
simple color patches where spatial influence and background factors, etc. are taken
into consideration.
1 Fundamentals of Color 13

1.7 Color Spaces: Linear and NonLinear Scales

Measurement of physical property is a very common activity in modern life.

Conveying the measured value by a scale number enables the quantitative descrip-
tion of certain property, such as length and mass. A uniform scale ensures that the
fundamental operations of algebra (addition, subtraction, equality, less than, greater
than, etc.) are applicable. It is therefore possible to apply mathematical manipulation
within such a scale system. In the meantime, establishing a perceptual color-
matching system is the first step toward color processing. Deriving a color scale
system (or color space) is the second step, which makes color image processing and
analysis a valid operation.
Establishing a color scale is complex because physical property is much easier
to be accessed than the sensation of human color perception. For example, a gray
scale with equal increment in a physical property like the reflectance factor (in 0.05
difference) is shown in Fig. 1.7a. It is obvious that in this scale the reflectance factor
does not yield an even increment in visual sensation. As stated by Fechner’s law—
the sensation increases linearly as a function of the logarithm of stimulus intensity
[4,7]—it is known that a certain nonlinear transformation is required to turn physical
stimulus intensity into the perceived magnitude of a stimulus. Based on this concept
the CIE in 1976 recommended two uniform color spaces, CIELAB and CIELUV.
The following is a brief description in computing the CIELAB values from the
reflectance value of an object.
Take the tristimulus values X, Y, Z from (1.3),

L∗ = 116 (Y/Yn)1/3 − 16
 
a∗ = 500 (X/Xn )1/3 − (Y/Yn )1/3 (1.7)
 
b∗ = 200 (Y/Yn )1/3 − (Z/Zn)1/3

where Y/Yn , X/Xn , and Z/Zn > 0.008856; more details in CIE 15.24
X, Y, and Z are tristimulus values of the object measured
Xn , Yn , and Zn are the tristimulus values of a reference white object

Fig. 1.7 Gray scales in physical and perceptual linear space: (a) a gray scale with a linear
increment of the reflectance factor (0.05) and (b) a gray scale with a visually linear increment
of the L* (Lightness) value in the CIELAB coordinate
14 M.J. Shyu and J. Parkkinen

L∗ is the visual lightness coordinate

a∗ is the chromatic coordinate ranged approximately from red to green
b∗ is the chromatic coordinate ranged approximately from yellow to blue
Important criteria in designing the CIELAB color space are making the coordi-
nates visually uniform and maintaining the opponent hue relationship according to
the human color sensation. This equal CIELAB L∗ increment is used to generate
the gray scale in Fig. 1.7b, where it turns out to appear as a much smoother
gradation than another gray scale with an equal reflectance factor increment shown
in Fig. 1.7a.
It is important to note that the linear scale of a physical property, like the equal
increment of the reflectance factor, does not yield linear visual perception. It is
necessary to perform a certain nonlinear transformation from the physical domain
to the perceptual domain which is perceived as a linear scale by the human visual
system. As more and more research is dedicated to color science and engineering,
it has been discovered that the human visual system can adjust automatically to
a different environment by various adaptation processes. What kind of nonlinear
processing is needed to predict human color image perception from measured
physical property under various conditions therefore definitely deserves intense
analysis and study, which is covered also in the following chapters. There are many
more color spaces and models, like S-CIELAB, CIECAM02, iCAM, and spectral
process models for various color imaging processing and analysis for specific
conditions.
As shown above, color can be treated in two ways: as a perceived property
by humans or as physical signal causing color detection in a detection system.
Color is very common in our daily life yet not directly accessible. Scientists have
derived mathematical models to define color properties. Engineers control devices
to generate different colors. Artists know how to express their emotions by various
colors. In a way, the study of color image processing and analysis is to bring
more use of color into our lives. As shown in Fig. 1.8, the various colors can be
interpreted as completeness in accumulating wisdom. No matter how complicated
is our practice of color imaging science and technology, making life interesting and
colorful is an ultimate joy.

1.8 Concluding Remarks

At the end of this chapter, we have a short philosophical discussion about color. In
general texts and discussions the term “color” is not used rigorously. It can mean the
human sensation, it can mean the color signal reflected from an object, and it can
be a property of an object itself. In the traditional color approach, it is connected to
the model of human color vision. Yet the same vocabulary is used when considering
the animal vision, although the animal (color) vision systems may vary very much
from that of human. In order to analyze and manage the color, we need to define
1 Fundamentals of Color 15

Fig. 1.8 Colorful banners are used in Japanese traditional buildings (Photographed by M. James
Shyu in Kyoto, Japan)

color well. In this chapter, and in the book, the spectral approach is described in
addition to the traditional color representation. In the spectral approach, color means
the color signal originated from the object and reaching the color detection system.
Both approaches are used in this book depending on the topic of the chapter.
In the traditional color science, black and white, and gray levels in between, are
called achromatic light. This means that they differ from each other only by radiant
intensity, or luminous intensity in photometrical terms. Other light is chromatic.
Hence, one may say that black, white, and gray are not colors. This is a meaningful
description only, if we have a fixed detection system, which is well defined. In the
traditional color approach, the human color vision is considered to be based on a
fixed detection system. Studies about human cone sensitivities and cone distribution
show that this is not the case [12].
In the spectral approach, the “achromaticity” of black, white, and gray levels
is not so obvious. In the spectral sense, the ultimate white is the equal energy
white, for which the spectrum intensity is a constant maximum value over the whole
wavelength range. When we start to decrease the intensity for some wavelength, the
spectrum changes and at a certain point the spectrum represents a color in traditional
means. If we consider white not to be a color, we have to define “epsilon” for each
wavelength by which the change from the white spectrum makes it a color. Also,
white spectrum can be seen as the limit of sequence of color spectra. This means
that in the traditional color approach, the limit of sequence of colors is not color.
Blackness, whiteness, and grayness are also dependent on detection system.
Detected signal looks white, when all the wavelength-sensitive sensors give the
16 M.J. Shyu and J. Parkkinen

a
1

EE white
Limited white

0
380 430 480 530 580 630 680 730 780

b
1

sensor A1
sensor A2
sensor A3

0
380 430 480 530 580 630 680 730 780

c
1

sensor B1
sensor B2
sensor B3

0
380 430 480 530 580 630 680 730 780

Fig. 1.9 White is a relative attribute. (a) Two spectra, equal energy white (blue line) and a
spectrum which look white for sensors A, but colored to sensors B (red line). (b) Sensors A,
sensitivity functions have the same shape as the “limited white” (red line) spectrum on (a).
(c) Sensors B, sensitivity functions does not match with spectrum “limited white” (red line) in (a)

maximum response. In Fig. 1.9a there are two color signals, which both “looks
white” for the theoretical color detection system given in Fig. 1.9b. But if we change
the detector system to one shown in Fig. 1.9c, the other white is a colored signal,
since not all the sensors have maximum input.
1 Fundamentals of Color 17

With this small discussion, we want to show that in the color science there is a
need and development into direction of generalized color. In this approach, color is
not restricted to the human visual system, but its basis is in a measurable and well-
defined color signal. Signal, which originates from the object, reaches the color
detection system, and carries the full color information of the object. The traditional
color approach is shown to be powerful tool to manage color for human vision.
The well defined models are useful tools also in the future, but the main restriction,
uncertainty in understanding of detection system, needs much research also in the
future.

References

1. ANSI CGATS.3–1993 (1993) Graphic technology—Spectral measurement and colorimetric

computation for graphic arts images. NPES
2. Berlin B, Kay P (1969) Basic color terms: their universality and evolution University of
California Press, Berkeley, CA
3. Berns R (2000) Billmeyer and Salzman’s principles of color technology 3rd edn. Wiley,
New York
4. Boynton RM (1984) Psychophysics, in optical radiation measurement. In: Bartleson CJ, Franc
Grum (eds) Visual measurement Vol 5 Academic Press p 342
5. Brehm PV (1992) Introduction to densitometry. Graphic Communications Association 3rd
Revision
6. Publication CIE No 15.2 (1986) Colorimetry. Second edition
7. Fechner G (1966) Elements of psychophysics. In: Adler HE, Howes DH, Boring EG (editors
and translators), Vol I Holt, New York
8. Foster DH (2003) Does colour constancy exist? Trends in Cognitive Sciences 7(10)439–443
9. John Gage (1995) Colour and culture: practice and meaning from antiquity to abstraction.
Thames and Hudson, Singapore
10. Goethe JW, Farbenlehre Z, Cotta T (1810) In: Goethe JWv, English version: Theory of Colors.
MIT, Cambridge, MA, USA, 1982
11. Heikkinen V (2011) Kernel methods for estimation and classification of data from spectral
imaging. PhD thesis, University of Eastern Finland, Finland
12. Hofer H, Carroll J Neitz J, Neitz M, Williams DR (2005) Organization of the human
trichromatic cone mosaic J Neurosci 25(42):9669–9679
13. Jacobs GH (1996) Primate photopigments and primate color vision. Proc Natl Acad Sci
93(2):577–581
14. Kuehni RG (2003) Color space and its divisions: color order from antiquity to the present
Wiley, Hoboken, NJ, USA
15. Land E (1977) The Retinex theory of color vision. Sci Am 237(6):108–128
16. Lenz R (2001) Estimation of illumination characteristics IEEE Trans Image Process
10(7):1031–1038
17. Maloney LT (1986) Evaluation of linear models of surface spectral reflectance with small
numbers of parameters J Opt Soc Am A 3:1673–1683
18. Maloney LT, Wandell B (1986) Color constancy: a method for recovering surface spectral
reflectance. J Opt Soc Am A 3:29–33
19. Menzel R, Backhaus W (1989) Color vision in honey bees: Phenomena and physiological
mechanisms. In: Stavenga D, Hardie N (eds) Facets of vision. Berlin 281–297
20. Sir Isaac Newton (1730) Opticks: or, a treatise of the reflections, refractions, inflections and
colours of light, 4th edn. Innys, London
18 M.J. Shyu and J. Parkkinen

21. George Palmer, Theory of Colors and Vision”, from Selected papers on Colorimetry—
Fundamentals. In D.L. MacAdam (ed.), SPIE Milestone Series Volume MS77, pp. 5–8, SPIE
Optical Engineering Press, 1993 (Originally printed by Leacroft 1777 reprinted from Sources
of Color Science, pp. 40–47, MIT Press, 1970)
22. Parkkinen JPS, Jaaskelainen T, Oja E (1985) Pattern recognition approach to color measure-
ment and discrimination Acta Polytechnica Scandinavica: Appl Phys 1(149):171–174
23. Parkkinen JPS, Hallikainen J, Jaaskelainen T (1989) Characteristic spectra of Munsell colors.
J Opt Soc Am A 6:318–322
24. Wade NJ (1999) A natural history of vision MIT Press, Cambridge, MA, USA 2nd printing
25. Wandell B (1985) The synthesis and analysis of color images, NASA Technical Memorandum
86844. Ames Research Center, California, USA, pp 1–34
26. William David Wright, The CIE Contribution to Colour Technology, 1931 to 1987, pp. 2–5, in
Inter-Society Color Council News, Number 368, July/August, 1997
27. Wyszecki G, Stiles W (1982) Color science: concepts and methods, quantitative data and
formulae 2nd edition Wiley, New York
28. Zollinger H (1999) Color: a multidisciplinary approach. Wiley, Weinheim
Chapter 2
CIECAM02 and Its Recent Developments

Ming Ronnier Luo and Changjun Li

The reflection is for the colors what the echo is for the sounds
Joseph Joubert

Abstract The development of colorimetry can be divided into three stages: colour
specification, colour difference evaluation and colour appearance modelling. Stage 1
considers the communication of colour information by numbers. The second stage
is colour difference evaluation. While the CIE system has been successfully applied
for over 80 years, it can only be used under quite limited viewing conditions,
e.g., daylight illuminant, high luminance level, and some standardised view-
ing/illuminating geometries. However, with recent demands on crossmedia colour
reproduction, e.g., to match the appearance of a colour or an image on a display
to that on hard copy paper, conventional colorimetry is becoming insufficient. It
requires a colour appearance model capable of predicting colour appearance across
a wide range of viewing conditions so that colour appearance modelling becomes
the third stage of colorimetry. Some call this as advanced colorimetry. This chapter
will focused on the recent developments based on CIECAM02.

Keywords Color appearance model • CAM • CIECAM02 • Chromatic adap-

tation transforms • CAT • Colour appearance attributes • Visual phenomena •
Uniform colour spaces

M.R. Luo ()

Zheijiang University, Hangzhou, China
University of Leeds, Leeds, UK
e-mail: [Link]@[Link]
C. Li
Liaoning University of Science and Technology, Anshan, China

C. Fernandez-Maloigne (ed.), Advanced Color Image Processing and Analysis, 19

DOI 10.1007/978-1-4419-6190-7 2,
© Springer Science+Business Media New York 2013
20 M.R. Luo and C. Li

2.1 Introduction

The development of colorimetry [1] can be divided into three stages: colour
specification, colour difference evaluation and colour appearance modelling. Stage 1
considers the communication of colour information by numbers. The Commission
Internationale de l’Eclairage (CIE) recommended a colour specification system in
1931 and later, it was further extended in 1964 [2]. The major components include
standard colorimetric observers, or colour matching functions, standard illuminants
and standard viewing and illuminating geometry. The typical colorimetric measures
are the tristimulus value (X,Y, Z), chromaticity coordinates (x, y), dominant wave-
length, and excitation purity.
The second stage is colour difference evaluation. After the recommendation
of the CIE specification system in 1931, it was quickly realised that the colour
space based on chromaticity coordinates was far from a uniform space, i.e., two
pairs of stimuli having similar perceived colour difference would show large
difference of the two distances from the chromaticity diagram. Hence, various
uniform colour spaces and colour difference formulae were developed. In 1976,
the CIE recommended CIELAB and CIELUV colour spaces [2] for presenting
colour relationships and calculating colour differences, More recently, the CIE
recommended the CIEDE2000 [3] for evaluating colour differences.
While the CIE system has been successfully applied for over 80 years, it can only
be used under quite limited viewing conditions, for example, daylight illuminant,
high luminance level, and some standardised viewing/illuminating geometries.
However, with recent demands on cross-media colour reproduction, for example,
to match the appearance of a colour or an image on a display to that on hard
copy paper, conventional colorimetry is becoming insufficient. It requires a colour
appearance model capable of predicting colour appearance across a wide range of
viewing conditions so that colour appearance modelling becomes the third stage of
colorimetry. Some call this as advanced colorimetry.
A great deal of research has been carried out to understand colour appearance
phenomena and to model colour appearance. In 1997, the CIE recommended a
colour appearance model designated CIECAM97s [4,5], in which the “s” represents
a simple version and the “97” means the model was considered as an interim model
with the expectation that it would be revised as more data and better theoretical un-
derstanding became available. Since then, the model has been extensively evaluated
by not only academic researchers but also industrial engineers in the imaging and
graphic arts industries. Some shortcomings were identified and the original model
was revised. In 2002, a new model: CIECAM02 [6, 7] was recommended, which is
simpler and has a better accuracy than CIECAM97s.
The authors previously wrote an article to describe the developments of
CIECAM97s and CIECAM02 [8]. The present article will be more focused on
the recent developments based on CIECAM02. There are six sections in this
chapter. Section 2.2 defines the viewing conditions and colour appearance terms
used in CIECAM02. Section 2.3 introduces some important colour appearance data
2 CIECAM02 and Its Recent Developments 21

sets which were used for deriving CIECAM02. In Sect. 2.4, a brief introduction
of different chromatic adaptation transforms (CAT) leading to the CAT02 [8],
embedded in CIECAM02, will be given. Section 2.5 gives various visual phenomena
predicted by CIECAM02. Section 2.6 summarises some recent developments of the
CIECAM02. For example, the new uniform colour spaces based on CIECAM02
by Luo et al. (CAM02-UCS, CAM02-SCD and CAM02-LCD) [9] will be covered.
Xiao et al. [10–12] extended CIECAM02 to predict the change in size of viewing
field on colour appearance, known as size effect. Fu et al. [13] has extended the
CIECAM02 for predicting colour appearances of unrelated colours presented in
mesopic region. Finally, efforts were paid to modify the CIECAM02 in connection
with international color consortium (ICC) profile connection space for the colour
management [14]. In the final section, the authors point out a concept of the
universal model based on CIECAM02.

2.2 Viewing Conditions and Colour Appearance Attributes

The step-by-step calculation of CIECAM02 is given in Appendix. In order to use

CIECAM02 correctly, it is important to understand the input and output parameters
of the model. Figure 2.1 shows the viewing parameters, which define the viewing
conditions, and colour appearance terms, which are predicted by the model. Each of
them will be explained in this section. Xw , Yw , Zw are the tristimulus values of the
reference white under the test illuminant; LA specifies the luminance of the adapting
field; Yb defines the luminance factor of background; the definition of surround will
be introduced in later this section.
The output parameters from the model include Lightness (J), Brightness (Q),
Redness–Greenness (a), Yellowness–Blueness (b), Colourfulness (M), Chroma (C),
Saturation (s), Hue composition (H), and Hue angle (h). These attributes will also
be defined in this section.

Fig. 2.1 A schematic diagram of a CIE colour appearance model

22 M.R. Luo and C. Li

Fig. 2.2 Configuration for

viewing colour patches of Ref. White
related colours
stimulus

Proximal field

Background

Surround

2.2.1 Viewing Conditions

The aim of the colour appearance model is to predict the colour appearance under
different viewing conditions. Various components in a viewing field have an impact
on the colour appearance of a stimulus. Hence, the accurate definition of each
component of the viewing field is important. Figures 2.2–2.4 are three configurations
considered in this chapter: colour patches for related colours, images for related
colours, and patches for unrelated colours. The components in each configuration
will be described below. Note that in the real world, objects are normally viewed
in a complex context of many stimuli; they are known as “related” colours. An
“unrelated colour” is perceived by itself, and is isolated, either completely or
partially, from any other colours. Typical examples of unrelated colours are signal
lights, traffic lights, and street lights, viewed in a dark night.

[Link] Stimulus

In Figs. 2.2 and 2.4 configurations, the stimulus is a colour element for which a
measure of colour appearance is required. Typically, the stimulus is taken to be
a uniform patch of about 2◦ angular subtense. A stimulus is first defined by the
tristimulus values (X,Y, Z) measured by a tele-spectroradiometer (TSR) and then
normalised against those of reference white so that Y is the percentage reflection
factor.
In Fig. 2.3 configuration, the stimulus becomes an image. The pixel of each
image is defined by device independent coordinates such as CIE XYZ or CIELAB
values.
2 CIECAM02 and Its Recent Developments 23

Fig. 2.3 Configuration for viewing images

Fig. 2.4 Configuration for

viewing unrelated colours

stimulus

Dark surround

[Link] Proximal Field

In Fig. 2.2 configuration, proximal field is the immediate environment of the

colour element considered, extending typically for about 2◦ from the edge of that
colour element in all or most directions. Currently, proximal field is not used in
CIECAM02. It will be applied when simultaneous contrast effect to be introduced
in the future.
This element is not considered in Figs. 2.3 and 2.4 configurations.

[Link] Reference White

In Fig. 2.2 configuration, the reference white is used for scaling lightness (see later)
of the test stimulus. It is assigned to have a lightness of 100. It is measured by a
TSR again to define the tristimulus values of the light source (XW ,YW , ZW ) in cd/m2
unit. The parameter of LW (equal to YW ) in the model defines the luminance of the
24 M.R. Luo and C. Li

light source. When viewing unrelated colours, there is no such element. For viewing
images, the reference white will be the white border (about 10 mm) surrounding the
image.
The reference white in this context can be considered as the “adopted white”
i.e., the measurement of “a stimulus that an observer who is adapted to the viewing
environment would judge to be perfectly achromatic and to have a reflectance factor
of unity (i.e., have absolute colorimetric coordinates that an observer would consider
to be the perfect white diffuser)” [ISO 12231] For viewing an image, there could
be some bright areas such as a light source or specularly reflecting white objects,
possibly illuminated by different sources. In the latter case, the “adapted white” (the
actual stimulus which an observer adapted to the scene judges to be equivalent to a
perfect white diffuser) may be different from the adopted white measured as above.

[Link] Background

In Fig. 2.2 configuration, background is defined as the environment of the colour

element considered, extending typically for about 10◦ from the edge of the proximal
field in all, or most directions. When the proximal field is the same colour as the
background, the latter is regarded as extending from the edge of the colour element
considered. Background is measured by a TSR to define background luminance, Lb .
In CIECAM02, background is defined by the luminous factor, Yb = 100 × Lb /LW .
There is no such element for Fig. 2.4 configuration, normally in complete
darkness. For viewing images (Fig. 2.3), this element can be the average Y value
for the pixels in the entire image, or frequently, a Y value of 20, approximate an L*
of 50 is used.

[Link] Surround

A surround is a field outside the background in Fig. 2.2 configuration, and outside
the white border (reference white) in Fig. 2.3. Surround includes the entire room or
the environment. Figure 2.4 configuration has a surround in complete darkness.
Surround is not measured directly, rather the surround ratio is determined and
used to assign a surround. The surround ratio, SR , can be computed:

SR = LSW /LDW , (2.1)

where LSW is the luminance of the surround white and LDW is the luminance of the
device white. LSW is a measurement of a reference white in the surround field while
LDW is a measurement of the device white point for a given device, paper or peak
white. If SR is 0, then a dark surround is appropriate. If SR is less than 0.2, then a dim
surround should be used while an SR of greater than or equal to 0.2 corresponds to
an average surround. Different surround “average,” “dim,” “dark” leads to different
2 CIECAM02 and Its Recent Developments 25

Table 2.1 Parameter settings for some typical applications

Ambient Scene or
illumination device Adopted
in lux (or white LA white
Example cd/m2 ) luminance in cd/m2 point SR Surround
Surface colour 1,000 (318.3) 318.30 cd/m2 60 Light booth 1 Average
evaluation in a
light booth
Viewing 38 (12) 80 cd/m2 20 Display 0.15 Dim
self-luminous and
display at home ambient
Viewing slides in 0 (0) 150 cd/m2 30 Projector 0 Dark
dark room
Viewing 500 (159.2) 80 cd/m2 15 Display 2 Average
self-luminous
display under
office
illumination

parameters (F: incomplete adaptation factor; Nc: chromatic induction factor and c:
impact of surround) used in CIECAM02. Table 2.1 define SR values in some typical
examples in real applications.

[Link] Adapting Field

For Fig. 2.2 configuration, adapting field is the total environment of the colour
element considered, including the proximal field, the background and the surround,
and extending to the limit of vision in all directions. For Fig. 2.3 image configura-
tion, it can be approximated the same as background, i.e., approximate an L∗ of 50.
The luminance of adapting field is expressed as LA , which can be approximated
by LW × Yb /100, or by Lb .

Photopic, Mesopic and Scotopic Vision

Another parameter is also very important concerning the range of illumination from
the source. It is well known that rods and cones in our eyes are not uniformly
distributed on the retina. Inside the foveola (the central 1◦ field of the eye), there
are only cones; outside, there are both cones and rods; in the area beyond about
40◦ from the visual axis, there are nearly all rods and very few cones. The rods
provide monochromatic vision under low luminance levels; this scotopic vision is
in operation when only rods are active, and this occurs when the luminance level is
less than about 0.1 cd/m2. Between this level and about 10 cd/m2 , vision involves a
mixture of rod and cone activities, which is referred to as mesopic vision. It requires
luminance of about 10 cd/m2 for photopic vision in which only cones are active.
26 M.R. Luo and C. Li

2.2.2 Colour Appearance Attributes

CIECAM02 predicts a range of colour appearance attributes. For each attribute,

it will be accurately defined mainly following the definitions of CIE International
Lighting Vocabulary [15]. Examples will be given to apply them in the real-world
situation, and finally the relationship between different attributes will be introduced.

[Link] Brightness (Q)

This is a visual perception according to which an area appears to exhibit more or

less light. This is an openended scale with a zero origin defining the black.
The brightness of a sample is affected by the luminance of the light source used.
A surface colour illuminated by a higher luminance would appear brighter than the
same surface illuminated by a lower luminance. This is known as “Steven Effect”
(see later).
Brightness is an absolute quantity, for example, a colour appears much brighter
when it is viewed under bright outdoor sunlight than under moonlight. Hence, their
Q values could be largely different.

[Link] Lightness (J)

This is the brightness of an area judged relative to the brightness of a similarly

illuminated reference white.
It is a relative quantity, for example, thinking a saturated red colour printed
onto a paper. The paper is defined as reference white having a lightness of 100.
By comparing the light reflected from both surfaces in the bright sunlight, the red
has a lightness of about 40% of the reference white (J value of 40). When assessing
the lightness of the same red colour under the moonlight against the same reference
white paper, the lightness remains more or less the same with a J of 40.
It can be expressed by J = QS /QW , where QS and QW are the brightness values
for the sample and reference white, respectively.

[Link] Colourfulness (M)

Colourfulness is that attribute of a visual sensation according to which an area

appears to exhibit more or less chromatic content.
This is an open-ended scale with a zero origin defining the neutral colours.
Similar to the brightness attribute, the colourfulness of a sample is also affected
by luminance. An object illuminated under bright sunlight would appear more
colourful than when viewed under moonlight, such as M value changes from 2000
to 1 with a ratio of 2000.
2 CIECAM02 and Its Recent Developments 27

Fig. 2.5 An image to illustrate saturation

[Link] Chroma (C)

This is the colourfulness of an area judged as a proportion of the brightness of a

similarly illuminated reference white. This is an open-ended scale with a zero origin
representing neutral colours. It can be expressed by C = M/QW .
The same example is given here, a saturated red printed on a white paper. It
has a colourfulness of 50 against the white paper having a brightness of 250 when
viewed under sunlight. When viewed under dim light, colourfulness reduces to 25
and brightness of paper also reduces to half. Hence, the C value remains unchanged.

[Link] Saturation (S)

This is the colourfulness of an area judged in proportion to its brightness as

expressed by s = M/Q, or s = C/J. This scale runs from zero, representing neutral
colours, with an open end.
Taking Figs. 2.3–2.5 as an example, the green grass under sunlight is bright and
colourful. In contrast, those under the tree appear dark and less colourful. Because
they are the same grass in the field, we know that they have the same colour, but their
brightness and colourfulness values are largely different. However, their saturation
values will be very close because it is the ratio between brightness and colourfulness.
Similar example can also be found in the image on the brick wall. Hence, saturation
could be a good measure for detecting the number and size of objects in an image.
28 M.R. Luo and C. Li

[Link] Hue (H and H)

Hue is the attribute of a visual sensation according to which an area appears to be

similar to one, or to proportions of two, of the perceived colours red, yellow, green
and blue.
CIECAM02 predicts hue with two measures: hue angle (h) ranging from 0◦
to 360◦ , and hue composition (H) ranging from 0, through 100, 200, 300, to 400
corresponding to the psychological hues of red, yellow, green, blue and back to red.
These four hues are the psychological hues, which cannot be described in terms of
any combinations of the other colour names. All other hues can be described as a
mixture of them. For example, an orange colour should be described as mixtures of
red and yellow, such as 60% of red and 40% of yellow.

2.3 Colour Appearance Data Sets

Colour appearance models based on colour vision theories have been developed to
fit various experimental data sets, which were carefully generated to study particular
colour appearance phenomena. Over the years, a number of experimental data sets
were accumulated to test and develop various colour appearance models. Data sets
investigated by CIE TC 1-52 CAT include: Mori et al. [16] from the Color Science
Association of Japan, McCann et al. [17] and Breneman [18] using a haploscopic
matching technique; Helson et al. [19], Lam and Rigg [20] and Braun and Fairchild
[21] using the memory matching technique; and Luo et al. [22, 23] and Kuo and
Luo [24] using the magnitude estimation method. These data sets, however, do
not include visual saturation correlates. Hence, Juan and Luo [25, 26] investigated
a data set of saturation correlates using the magnitude estimation method. The
data accumulated played an important role in the evaluation of the performance
of different colour appearance models and the development of the CIECAM97s and
CIECAM02.

2.4 Chromatic Adaptation Transforms

Arguably, the most important function of a colour appearance model is chromatic

adaptation transform. CAT02 is the chromatic adaptation transformation imbedded
in CIEAM02. This section covers the developments towards this transform.
Chromatic adaptation has long been extensively studied. A CAT is capable of
predicting corresponding colours, which are defined as pairs of colours that look
alike when one is viewed under one illuminant (e.g., D651 ) and the other is under

1 In
this chapter we will use for simplified terms “D65” and “A” instead of the complete official
CIE terms: “CIE standard illuminant D65” and “CIE standard illuminant A”.
2 CIECAM02 and Its Recent Developments 29

a different illuminant (e.g., A). The following is divided into two parts: light and
chromatic adaptation, and the historical developments of Bradford transform [20],
CMCCAT2000 [27] and CAT02.

2.4.1 Light and Chromatic Adaptation

Adaptation can be divided into two: light and chromatic. The former is the
adaptation due to the change of light levels. It can be further divided into two: light
adaptation and dark adaptation. Light adaptation is the decrease in visual sensitivity
upon an increase in the overall level of illumination. An example occurs when
entering a bright room from a dark cinema. Dark adaptation is opposite to light
adaptation and occurs, for example, when entering a dark cinema from a well-lit
room.

2.4.2 Physiological Mechanisms

The physiology associated with adaptation mainly includes rod–cone transition,

pupil size (dilation and constriction), receptor gain and offset. As mentioned earlier,
the two receptors (cones and rods) functioning entirely for photopic (above approx-
imately 10 cd/m2 ) and for scotopic (below approximately 0.01 cd/m2 ), respectively.
Also, both are functioning in mesopic range between the two (approximately from
0.01 cd/m2 to 10 cd/m2 ).
The pupil size plays an important role in adjusting the amount of light that
enters the eye by dilating or constricting the pupil: it is able to adjust the light by a
maximum factor of 5. During dark viewing conditions, the pupil size is the largest.
Each of the three cones responds to light in a nonlinear manner and is controlled by
the gain and inhibitory mechanisms.
Light and dark adaptations only consider the change of light level, not the
difference of colour between two light sources (up to the question of Purkinje
shift due to the difference in the spectral sensitivity of the rods and cones).
Under photopic adaptation conditions, the difference between the colours of two
light sources produces chromatic adaptation. This is responsible for the colour
appearance of objects, and leads to the effect known as colour constancy (see also
Chap. 2: Chromatic constancy). The effect can also be divided into two stages: a
“chromatic shift” and an “adaptive shift”. Consider, for example, what happens
when entering a room lit by tungsten light from outdoor daylight. We experience
that all colours in the room instantly become reddish reflecting the relative hue of
the tungsten source. This is known as the “colorimetric shift” and it is due to the
operation of the sensory mechanisms of colour vision, which occur because of the
changes in the spectral power distribution of the light sources in question. After a
certain short adaptation period, the colour appearances of the objects become more
30 M.R. Luo and C. Li

normal. This is caused by the fact that most of coloured objects in the real world
are more or less colour constant (they do not change their colour appearance under
different illuminants). The most obvious example is white paper always appears
white regardless of which illuminant it is viewed under. The second stage is called
the “adaptive shift” and it is caused by physiological changes and by a cognitive
mechanism, which is based upon an observer’s knowledge of the colours in the
scene content in the viewing field. Judd [28] stated that “the processes by means of
which an observer adapts to the illuminant or discounts most of the effect of non-
daylight illumination are complicated; they are known to be partly retinal and partly
cortical”.

2.4.3 Von Kries Chromatic Adaptation

The von Kries coefficient law is the oldest and widely used to quantify chromatic
adaptation. In 1902, von Kries [29] assumed that, although the responses of the three
cone types (RGB)2 are affected differently by chromatic adaptation, the spectral
sensitivities of each of the three cone mechanisms remain unchanged. Hence,
chromatic adaptation can be considered as a reduction of sensitivity by a constant
factor for each of the three cone mechanisms. The magnitude of each factor depends
upon the colour of the stimulus to which the observer is adapted. The relationship,
given in (2.2), is known as the von Kries coefficient law.
Rc = α · R,
Gc = β · G,
Bc = γ · B, (2.2)
where Rc , Gc , Bc and R, G, B are the cone responses of the same observer, but
viewed under test and reference illuminants, respectively. α , β and γ are the von
Kries coefficients corresponding to the reduction in sensitivity of the three cone
mechanisms due to chromatic adaptation. These can be calculated using (2.3).
     
Rwr Gwr Bwr
α= ; β= ; γ= , (2.3)
Rw Gw Bw
where
R Rc G Gc B Bc
= , = , = , (2.4)
Rw Rwr Gw Gwr Bw Bwr

2 Inthis chapter the RGB symbols will be used for the cone fundamentals, in other chapters the
reader will find the LMS symbols. The use of RGB here should not be confused with the RGB
primaries used in visual colour matching.
2 CIECAM02 and Its Recent Developments 31

Here Rwr , Gwr , Bwr , and Rw , Gw , Bw are the cone responses under the reference and
test illuminants, respectively. Over the years, various CATs have been developed but
most are based on the von Kries coefficient law.

2.4.4 Advanced Cats: Bradford, CMCCAT20000 and CAT02

In 1985, Lam and Rigg accumulated a set of corresponding colour pairs. They used
58 wool samples that had been assessed twice by a panel of five observers under D65
and A illuminants. The memory-matching technique was used to establish pairs of
corresponding colours. In their experiment, a subgroup of colours was first arranged
in terms of chroma and hue, and each was then described using Munsell H V/C
coordinates. The data in H V/C terms, were then adjusted and converted to CIE
1931 XYZ values under illuminant C. Subsequently, the data under illuminant C
were transformed to those under illuminant D65 using the von Kries transform.
They used this set of data to derive a chromatic transform known as BFD transform
now. The BFD transform can be formulated as the following:

[Link] Bfd Transform [20]

Step 1:
⎛ ⎞ ⎛ ⎞ ⎛ ⎞
R X 0.8951 0.2664 0.1614
1
⎝ G ⎠ = MBFD ⎝ Y ⎠ with MBFD = ⎝ −0.7502 1.7135 0.0367 ⎠ .
Y
B Z 0.0389 −0.0685 1.0296

Step 2:
⎛ ⎞ ⎛ ⎞⎛ ⎞
Rc Rwr /Rw R
⎝ Gc ⎠ = ⎝ Gwr /Gw ⎠⎝ G ⎠ with
p
Bc Bwr /Bw sign(B)|B| p

p = (Bw /Bwr )0.0834 .

Step 3:
⎛ ⎞ ⎛ ⎞
Xc Y Rc
⎝ Yc ⎠ = M −1 ⎝ Y Gc ⎠ .
BFD
Zc Y Bc
Note that the BFD transform is a nonlinear transform. The exponent p
in step 2 for calculating the blue corresponding spectral response can be
considered as a modification of the von Kries type of transform. The BFD
transform performs much better than the von Kries transform. In 1997, Luo
32 M.R. Luo and C. Li

and Hunt [30] in 1997 modified the step 2 in the above BFD transform by
introducing an adaptation factor D. The new step 2 becomes,
Step 2’
⎛ ⎞ ⎛ ⎞
Rc [D(Rwr /Rw ) + 1 − D]R
⎝ Gc ⎠ = ⎝ [D(Gwr /Gw ) + 1 − D]G ⎠,
Bc [D(Bwr /Bwp ) + 1 − D]sign(B)|B| p
where
1/4
D = F − F/[1 + 2LA + L2A /300].

The transform consisting of Step 1, Step 2’ and Step 3 was then recommended by
the colour measurement committee (CMC) of the society of dyers and colourists
(SDC) and, hence, was named as the CMCCAT97. This transform is included
in the CIECAM97s for describing colour appearance under different viewing
conditions. The BFD transform was originally derived by fitting only one data set,
Lam and Rigg. Although it gave a reasonably good fit to many other data sets,
it predicted badly the McCann data set. In addition, the BFD and CMCCAT97
include an exponent p for calculating the blue corresponding spectral response. This
causes uncertainty in reversibility and complexity in the reverse mode. Li et al.
[31] addressed this problem and provided a solution by including an iterative
approximation using the Newton method. However, this is unsatisfactory in imaging
applications where the calculations need to be repeated for each pixel. Li et al.
[27] gave a linearisation version by optimising the transform to fit all the available
data sets, rather than just the Lam and Rigg set. The new transform, named
CMCCAT2000, is given below.

[Link] Cmccat2000

Step 1:
⎛ ⎞ ⎛ ⎞ ⎛ ⎞
R X 0.7982 0.3389 −0.1371
⎝ G ⎠ = M00 ⎝ Y ⎠ with M00 = ⎝ −0.5918 1.5512 0.0406 ⎠ .
B Z 0.0008 0.0239 0.9753

Step 2:
⎛ ⎞ ⎛ ⎞
Rc [D(Yw /Ywr )(Rwr /Rw ) + 1 − D]R
⎝ Gc ⎠ = ⎝ [D(Yw /Ywr )(Gwr /Gw ) + 1 − D]G ⎠
Bc [D(Yw /Ywr )(Bwr /Bw ) + 1 − D]B
with

D = F{0.08 log10 [0.5(LA1 + LA2 )] + 0.76 − 0.45(LA1 − LA2 )/(LA1 + LA2 )}.
2 CIECAM02 and Its Recent Developments 33

Step 3:
⎛ ⎞ ⎛ ⎞
Xc Rc
⎝ Yc ⎠ = M −1 ⎝ Gc ⎠ .
00
Zc Bc

The CMCCAT2000 not only overcomes all the problems with respect to reversibility
discussed above, but also gives a more accurate prediction than other transforms of
almost all the available data sets.
During and after the development of the CMCCAT2000, scientists decided to
drop the McCann et al. data set because the experiment was carried out under a
very chromatic adapting illuminant. Its viewing condition is much different from all
the other corresponding data sets. Hence, it would be better to optimising the linear
chromatic adaptation transform via fitting all the corresponding data sets without
the McCann et al. data set. The new matrix obtained by the authors, now named the
CAT02 matrix, is given by
⎛ ⎞
0.7328 0.4296 −0.1624
M02 = ⎝ −0.7036 1.6975 0.0061 ⎠ ,
0.0030 0.0136 0.9834

which was first included in the appendix of our paper [32] in 2002. At the same
time, Nathan Moroney (Chair of CIETC8-01 at that time) proposed a new formula
for D function:
 
1 −LA −42
D = F 1− e 92 . (2.5)
3.6

The CMCCAT2000 with the new matrix and D formula given by (2.5) becomes the
CAT02.
At a later stage, CIE TC 8-01 Colour Appearance Modelling for Colour
Management Systems had to choose a linear chromatic transform for CIECAM02.
Multiple candidates such as CMCCAT2000 [27], the sharp chromatic transform
[33] developed by Finlayson et al., and CAT02 [6–8] were proposed for use as
a von Kries type transform. All had similar levels of performance with respect
to the accuracy of predicting various combinations of previously derived sets of
corresponding colours. In addition to the sharpening of the spectral sensitivity
functions, considerations used to select the CIE transform included the degree
of backward compatibility with CIECAM97s and error propagation properties by
combining the forward and inverse linear CAT, and the data sets which were
used during the optimisation process. Finally, CAT02 was selected because it is
compatible with CMCCAT97 and was optimised using all available data sets except
the McCann et al. set, which includes a very chromatic adapting illuminant.
Figure 2.6 illustrates 52 pairs of corresponding colours predicted by CIECAM02
(or its chromatic adaptation transform, CAT02) from illuminant A (open circles
of vectors) to SE (open ends of vectors) plotted in the CIE u v chromaticity
diagram for the 2◦ observer. The open circle colours have a value of L∗ equal
34 M.R. Luo and C. Li

Fig. 2.6 The corresponding colours predicted by the CIECAM02 from illuminant A (open circles
of vectors) to illuminant SE (open ends of vectors) plotted in CIE u v chromaticity diagram for the
CIE 1931 standard colorimetric observer. The plus (+) and the dot (•) represent illuminants A and
SE , respectively

to 50 according to CIELAB under illuminant A. These were then transformed

by the model to the corresponding colours under illuminant SE (the equi-energy
illuminant). Thus, the ends of each vector represent a pair of corresponding colours
under the two illuminants. The input parameters are (the luminance of adapting
field) LA = 63.7 cd/m2 and average surround. The parameters are defined in the
Appendix.
The results show that there is a systematic pattern, i.e., for colours below v equal
to 0.48 under illuminant A the vectors are predicted towards the blue direction under
the illuminant SE . For colours outside the above region, the appearance change is in
a counterclockwise direction, i.e., red colours shift to yellow, yellow to green and
green to cyan as the illuminant changes from A to SE .

2.5 Colour Appearance Phenomena

This section describes a number of colour appearance phenomena studied by various

researchers in addition to the chromatic adaptation as described in the earlier
section. The following effects are also well understood.

2.5.1 Hunt Effect

Hunt [34] studied the effect of light and dark adaptation on colour perception
and collected data for corresponding colours via a visual colorimeter using the
haploscopic matching technique, in which each eye was adapted to different viewing
conditions and matches were made between stimuli presented in each eye.
2 CIECAM02 and Its Recent Developments 35

The results revealed a visual phenomena known as Hunt effect [34]. It refers to
the fact that the colourfulness of a colour stimulus increases due to the increase
of luminance. This effect highlights the importance of considering the absolute
luminance level in colour appearance models, which is not considered in traditional
colorimetry.

2.5.2 Stevens Effect

Stevens and Stevens [35] asked observers to make magnitude estimations of the
brightness of stimuli across various adaptation conditions. The results showed
that the perceived brightness contrast increased with an increase in the adapting
luminance level according to a power relationship.

[Link] Surround Effect

Bartleson and Breneman [36] found that the perceived contrast in colourfulness
and brightness increased with increasing illuminance level from dark surround, dim
surround to average surround. This is an important colour appearance phenomenon
to be modelled, especially for the imaging and graphic arts industries where, on
many occasions, it is required to reproduce images on different media under quite
distinct viewing conditions.

2.5.3 Lightness Contrast Effect

The lightness contrast effect [37] reflects that the perceived lightness increases when
colours are viewed against a darker background and vice versa. It is a type of
simultaneous contrast effect considering the change of colour appearance due to
different coloured backgrounds. This effect has been widely studied and it is well
known that a change in the background colour has a large impact on the perception
of lightness and hue. There is some effect on colourfulness, but this is much smaller
than the effect on lightness and hue [37].

2.5.4 Helmholtz–Kohlrausch Effect

The Helmholtz–Kohlrausch [38] effect refers to a change in the brightness of colour

produced by increasing the purity of a colour stimulus while keeping its luminance
constant within the range of photopic vision. This effect is quite small compared
with others and is not modelled by CIECAM02.
36 M.R. Luo and C. Li

2.5.5 Helson–Judd Effect

When a grey scale is illuminated by a light source, the lighter neutral stimuli
will exhibit a certain amount of the hue of the light source and the darker
stimuli will show its complementary hue, which is known as the Helson–Judd
effect [39]. Thus for tungsten light, which is much yellower than daylight, the
lighter stimuli will appear yellowish, and the darker stimuli bluish. This effect is
not modelled by CIECAM02.

2.6 Recent Developments of CIECAM02

Recently, several extensions to the CIECAM02 have been made, which have
widened the applications of the CIECAM02. In this section, the extensions for
predicting colour discrimination data sets, size effects and unrelated colour appear-
ance in the mesopic region. Besides, recent developments from CIETC8-11 will be
reported as well.

2.6.1 CIECAM02-Based Colour Spaces

CIECAM02 [6, 7] includes three attributes in relation to the chromatic content:

chroma (C), colourfulness (M) and saturation (s). These attributes together with
lightness (J) and hue angle (h) can form three colour spaces: J, aC , bC , J, aM , bM
and J, as , bs where
aC = C · cos(h) aM = M · cos(h) as = s · cos(h)
, ,
bC = C · sin(h) bM = M · sin(h) bs = s · sin(h).
It was also found [40] that the CIECAM02 space is more uniform than the CIELAB
space. Thus, the CIECAM02 space is used as a connection space for the gamut
mapping in the colour management linked with the ICC profile [41, 42]. Further
attempts have been also made by the authors to extend CIECAM02 for predicting
available colour discrimination data sets, which include two types, for Large and
Small magnitude Colour Differences, designated by LCD and SCD, respectively.
The former includes six data sets with a total 2,954 pairs, having an average 10
ΔEab∗ units over all the sets. The SCD data with a total of 3,657 pairs having an

average 2.5 ΔEab ∗ units, are a combined data set used to develop the CIE 2000 colour

difference formula: CIEDE20003.

Li et al. [43] found that a colour space derived using J, aM , bM gave the most
uniform result when analysed using the large and small colour difference data sets.
Hence, various attempts [9, 43] were made to modify this version of CIECAM02
to fit all available data sets. Finally, a simple, generic form, (2.6) was found that
2 CIECAM02 and Its Recent Developments 37

Table 2.2 The coefficients

for CAM02-LCD, Versions CAM02 -LCD CAM02-SCD CAM02-UCS
CAM02-SCD and KL 0.77 1.24 1.00
CAM02-UCS c1 0.007 0.007 0.007
c2 0.0053 0.0363 0.0228

adequately fitted all available data.

(1 + 100 · c1) · J
J = ,
1 + c1 · J
M  = (1/c2 ) · ln(1 + c2 · M), (2.6)

where c1 and c2 are constants given in Table 2.2.

The corresponding colour space is J  , aM , bM where aM = M  · cos(h), and
bM = M  · sin(h). The colour difference between two samples can be calculated in


J  , aM , bM space using (2.7).

ΔE  = (ΔJ  /KL )2 + Δa2M + Δb 2M , (2.7)

where ΔJ  , ΔaM and ΔbM are the differences of J  , aM and bM between the “standard”
and “sample” in a pair. Here, KL is a lightness parameter and is given in Table 2.2.
Three colour spaces named CAM02-LCD, CAM02-SCD and CAM02-UCS were
developed for large, small and combined large and small differences, respectively.
The corresponding parameters in (2.6) and (2.7) are listed in Table 2.2.
The three new CIECAM02 based colour spaces, together with the other spaces
and formulae were also tested by Luo et al. [9]. The results confirmed that CAM02-
SCD and CAM02-LCD performed the best for small and large colour difference data
sets. When selecting one UCS to evaluate colour differences across a wide range,
CAM02-UCS performed the second best across all data sets. The authors have been
recommending using CAM02-UCS for all applications.
Figure 2.7 shows the relationship between CIECAM02 J and CAM02-UCS J’ and
Fig. 2.8 shows the relationship between CIECAM02 M and CAM02-UCS M’. It can
be seen that CIECAM02 J is less than CAM02-UCS J’ except at the two ends, while
CIECAM02 M is greater than CAM02-UCS M’ except when M = 0. Thus in order
to have a more uniform space, CIECAM02 J should be increased and CIECAM02
M should be decreased.
The experimental colour discrimination ellipses used in the previous studies
[44, 45] were also used for comparing different colour spaces. Figures 2.9 and
2.10 show the ellipses plotted in CIELAB and CAM02-UCS spaces, respectively.
The size of the ellipse was adjusted by a single factor in each space to ease
visual comparison. For perfect agreement between the experimental results and a
uniform colour space, all ellipses should be constant radius circles. Overall, it can
be seen that the ellipses in CIELAB (Fig. 2.9) are smaller in the neutral region
and gradually increase in size as chroma increases. In addition, the ellipses are
38 M.R. Luo and C. Li

Fig. 2.7 The full line shows

the relationship between J
and J  and the dotted line is
the 45◦ line

Fig. 2.8 The full line shows

the relationship between M
and M and the dotted line is
the 45◦ line

orientated approximately towards the origin except for those in the blue region in
CIELAB space. All ellipses in CAM02-UCS (Fig. 2.10) are approximately equal-
sized circles. In other words, the newly developed CAM02-UCS is much more
uniform than CIELAB.
2 CIECAM02 and Its Recent Developments 39

Fig. 2.9 Experimental

chromatic discrimination
ellipses plotted in CIELAB

Fig. 2.10 Experimental

chromatic discrimination
ellipses plotted in
CAM02-UCS

2.6.2 Size Effect Predictions Based on CIECAM02

The colour size effect is a colour appearance phenomenon [10–12], in which

the colour appearance changes according to different sizes of the same colour
stimulus. The CIE 1931 (2◦ ) and CIE 1964 (10◦ ) standard colorimetric observers
were recommended by the CIE to represent human vision in smaller and larger
than 4◦ viewing fields, respectively [2]. However, for a colour with a large size,
such as over 20◦ viewing field, no standard observer can be used. The current
40 M.R. Luo and C. Li

Fig. 2.11 The flow chart of size effect correction model based on CIECAM02

CIECAM02 is capable of predicting human perceptual attributes under various

viewing conditions. However, it cannot predict the colour size effect. The size effect
has been interested in many applications. For example, in the paint industry, the
paints purchased in stores usually do not appear the same comparing between those
shown in the packaging and painted onto the walls in a real room. This also causes
great difficulties for homeowners, interior designers and architects when they select
colour ranges. Furthermore, the display size tends to become larger. Colour size
effect has also been greatly interested by display manufacturers in order to precisely
reproduce or to enhance the source images on different sizes of colour displays.
With the above problems in mind, the CIE established a technical committee,
TC1-75, A comprehensive model for colour appearance with one of aims to take
colour size effect into account in the CIECAM02 colour appearance model [7].
In the recent work of Xiao et al. [10–12], six different sizes from 2◦ to 50◦ of
same colours were assessed by a panel of observers using colour-matching method
to match surface colours using a CRT display. The colour appearance data were
accumulated in terms of CIE tristimulus values. A consistent pattern of colour
appearance shifts was found according to different sizes for each stimulus. The
experimental results showed that attributes of lightness and chroma increase with
the increase of the physical size of colour stimulus. But the hue (composition) is not
affected by the change of physical size of colour stimulus. Hence, a model based on
CIECAM02 for predicting the size effect was derived. The model has the general
structure shown in Fig. 2.11.
Step 1 calculates or measures tristimulus values X, Y, Z of a 2◦ stimulus size
under a test illuminant XW ,YW , ZW , and provides a target stimulus size θ; next, Step
2 predicts the appearance attributes J,C and H using CIECAM02 for colours with 2◦
stimulus size; and Step 3 computes the scaling factors KJ and KC via the following
formulae:
KJ = −0.007θ + 1.1014,
KC = 0.008θ + 0.94.
2 CIECAM02 and Its Recent Developments 41

Fig. 2.12 The size effect

corrected attributes J  vs
CIECAM02 J under viewing
angles being 25◦ , (thick solid
line), 35◦ (dotted line) and
45◦ (dashed line),
respectively. The thin solid
line is the 45◦ line where
J = J 

Finally in Step 4, the colour appearance attributes J  , C and H  for the target
stimulus size θ are predicted using the formulae:

J  = 100 + KJ × (J − 100), (2.8)

C = KC × C, (2.9)

H  = H. (2.10)

The earlier experimental results [10] were used to derive the above model.
Figure 2.12 shows the corrected attributes J  of 25◦ , 35◦ and 45◦ , respectively,
plotted against J at 2◦ viewing field. The thick solid line is the corrected J  when
viewing field is 25◦ ; the dotted line corresponds to the J  with viewing angle being
35◦ . The dashed line is the J  with viewing angle of 45◦ . The thin solid line is the
45◦ line where J = J  . The trend is quite clear as shown in Fig. 2.12, i.e., an increase
of lightness for a larger viewing field. For example, when J = 60 with a size of 2◦ , J 
values are 62.9, 65.7 and 68.5 for sizes of 25◦ , 35◦ and 45◦ , respectively. However,
when J = 10 with a size of 2◦ , J  s become 16.6, 22.9 and 29.2 for 25◦ , 35◦ and 45◦ ,
respectively. This implies that the large effect is mainly occurred for the dark colour
region.
Figure 2.13 shows the corrected attributes C of 25◦ , 35◦ and 45◦, respectively
plotted against C at 2◦ viewing field. Vertical axis is the size effect corrected C .
The thick solid line is the corrected C when viewing angle is 25◦ ; the dotted line
corresponds to the C with viewing angle being 35◦ . The dashed line is the C with
viewing angle of 45◦ . The thin solid line is the 45◦ line where C = C . Again, a clear
trend in Fig. 2.13 is shown that an increase of chroma for a larger viewing field. For
example, when C is 60 with a size of 2◦ , C values are 68.4, 73.2 and 78.0 for sizes
42 M.R. Luo and C. Li

Fig. 2.13 The size effect

corrected attributes C vs
CIECAM02 C under viewing
being 25◦ , (thick solid line),
35◦ (dotted line) and 45◦
(dashed line), respectively.
The thin solid line is the 45◦
line where C = C

of 25◦ , 35◦ and 45◦ , respectively. However, when C is 10 with a size of 2◦ , C s
become 11.4, 12.2 and 13.0 for 25◦ , 35◦ and 45◦, respectively. This implies that the
large effect in mainly occurrs in the high chroma region.

2.6.3 Unrelated Colour Appearance Prediction Based

on CIECAM02

As mentioned at the beginning of this chapter, unrelated colours are important in

relation to safety issues (such as night driving). It includes signal lights, traffic lights
and street lights, viewed on a dark night. These colours are important in connection
with safety issues. The CIECAM02 was derived for predicting colour appearance
for related colours and it cannot be used for predicting unrelated colour appearance.
The CAM97u derived by Hunt [46] can be used for predicting unrelated colour
appearance. However, the model was not tested since there was no available visual
data for unrelated colours. Fu et al. [13] carried out the research work recently.
They accumulated a set of visual data using the configuration in Fig. 2.4. The data
were accumulated for the colour appearance of unrelated colours under photopic
and mesopic conditions. The effects of changes in luminance level and stimulus
size on appearance were investigated. The method used was magnitude estimation
of brightness, colourfulness and hue. Four luminance levels (60, 5, 1 and 0.1, cd/m2 )
were used. For each of the first three luminance levels, two stimulus sizes (10◦ ,
2◦ , 1◦ and 0.5◦ ) were used. Ten observers judged 50 unrelated colours. A total of
17,820 estimations were made. The observations were carried out in a completely
darkened room, after about 20 min adaptation; each test colour was presented on
2 CIECAM02 and Its Recent Developments 43

its own. Brightness and colourfulness were found to decrease with decreases of
both luminance level and stimulus size. The results were used to further extend
CIECAM02 for predicting unrelated colours under both photopic and mesopic
conditions. The model includes parameters to reflect the effects of luminance level
and stimulus size. The model is described below:

Inputs:

Measure or calculate the luminance L and chromaticity x,y of the test colour stimu-
lus corresponding to CIE colour-matching functions (2◦ or 10◦ ). The parameters are
the same as CIECAM02 except that the test illuminant is equal energy illuminant
(SE , i.e., XW = YW = ZW = 100), and LA = 1/5 of the adapting luminance, and the
surround parameters are set as those under the dark viewing condition. As reported
by Fu et al. [13], when there is no reference illuminant to compare with (such as
assessing unrelated colours), SE illuminant can be used by assuming no adaptation
takes place for unrelated viewing condition.
Step 1: Using the CIECAM02 (Steps 0–8, Step 10, ignore the calculation of Q and
s) to predict the (cone) achromatic signal A, colourfulness (M) and hue
(HC ).
Step 2: Modify the achromatic signal A since there is a contribution from rod
response using the formula:

Anew = A + kAAS with AS = (2.26L)0.42 .

Here, kA depends on luminance level and viewing angle size of the colour
stimulus.
Step 3: Modify the colourfulness M predicted from CIECAM02 using the follow-
ing formula:
Mnew = kM M.

Here, kM depends on luminance level and viewing angle size of the colour
stimulus.
Step 4: Predict the new brightness using the formula:

Qnew = Anew + Mnew /100.

Outputs: Brightness Qnew , colourfulness Mnew and hue composition HC .

Note that the hue composition HC is the same as predicted by CIECAM02. The
above model was tested using the visual data [13].
Figure 2.14 shows the brightness and colourfulness changes for a red colour of
medium saturation (relative to SE , huv = 355◦ , and suv = 1.252) as predicted by the
new model under different luminance levels. The luminance levels were varied from
44 M.R. Luo and C. Li

Fig. 2.14 The brightness and

colourfulness predicted by the
new model for a sample
varying in luminance level
with 2◦ stimulus size

Fig. 2.15 The brightness and

colourfulness predicted by the
new model for a sample
varying in stimulus size at 0.1
cd/m2 luminance level

0.01 to 1000 cd/m2 , and LA was set at one fifth of these values. The ratio Yb /Yw set
at 0.2. Figure 2.15 shows the brightness and colourfulness changes, for the same red
colour, predicted by the new model for different stimulus sizes ranging from 0.2◦
to 40◦ . The luminance level (L) was set at 0.1 cd/m2 . It can be seen that brightness
and colourfulness increase when luminance increases up to around 100 cd/m2 , and
they also increase when stimulus size increases. These trends reflect the phenomena
found in Fu et al.’s study, i.e. when luminance level increases, colours become
brighter and more colourful, and larger colours appear brighter and more colourful
than smaller sized colours; however, below a luminance of 0.1 cd/m2 and above a
luminance of 60 cd/m2 , and below a stimulus size of 0.5◦ and above a stimulus size
of 100 , these results are extrapolations, and must be treated with caution.
2 CIECAM02 and Its Recent Developments 45

2.6.4 Problems with CIECAM02

Since the recommendation of the CIECAM02 colour appearance model [6, 7] by

CIE TC8-01 Colour appearance modelling for colour management systems, it has
been used to predict colour appearance under a wide range of viewing conditions,
to specify colour appearance in terms of perceptual attributes, to quantify colour
differences, to provide a uniform colour space and to provide a profile connection
space for colour management. However, some problems have been identified and
various approaches have been proposed to repair the model to enable it to be used
in practical applications. During the 26th session of the CIE, held in Beijing in July
2007, a Technical Committee, TC8-11 CIECAM02 Mathematics, was formed to
modify or extend the CIECAM02 model in order to satisfy the requirements of a
wide range of industrial applications. The main problems that have been identified
can be summarised as follows:
1. Mathematical failure for certain colours
2. The CIECAM02 colour domain is smaller than that of ICC profile connection
space
3. The HPE matrix
4. The brightness function
Each problem will be reviewed in turn and then a possible solution that either repairs
the problem or extends the model will be given as well. Note that all notations used
in this paper have the same meaning as those in CIE Publication 159 [7].

[Link] Mathematical Failure

It has been found that the Lightness function:

J = 100(A/Aw)cz

gives a problem for some colours. In fact Li and Luo [47] have shown that Aw > 0,
but for some colours, the achromatic signal
 
A = 2Ra + Ga + (1/20)Ba − 0.305 Nbb

can be negative; thus, the ratio in the bracket for the J function is negative which
gives problem when computing J. At the beginning, it has been suggested that the
source of the problem is the CAT02 transform which, for certain colours, predicts
negative tristimulus values. Several approaches have been made on modifying the
CAT02 matrix. Brill and Süsstrunk [48–50] found that the red and green CAT02
primaries lie outside the HPE triangle and called this as the “Yellow-Blue” problem.
They suggested that the last row of the CAT02 matrix can be changed to 0, 0, 1. The
changed matrix is denoted by MBS . It has been found that for certain colours, using
46 M.R. Luo and C. Li

matrix MBS works well, but using matrix M02 does not. However, this repair seems
to correct neither the prediction of negative tristimulus values for the CAT02 nor the
failure of CIECAM02.
Another suggestion is equivalent to set Ra ≥ 0.1, i.e., if Ra < 0.1, then set Ra =
0.1, if Ra ≥ 0.1, then Ra does not change. Similar considerations are applied to
Ga and Ba . Thus, under this modification, the achromatic signal A is non-negative.
However, this change causes new problem with the inverse model.
Li et al. [51] gave a mathematical approach for obtaining CAT02 matrix.
The approach has two constraints. The first one is to ensure the CAT02 predict
corresponding colours with non-negative tristimulus values under all the illuminants
considered for all colours located on or inside the CIE chromaticity locus. The
second one is to fit all the corresponding colour data sets. This approach indeed
ensures the CAT02 with the new matrix predicts corresponding colours with non-
negative tristimulus values which is important in many applications. However, this
approach does not solve the mathematical failure problem for the CIECAM02.
Recently, Li et al. [14] proposed a mathematical approach for ensuring the
achromatic signal A being non-negative, at the same time the CIECAM02 should
fit all the colour appearance data sets. Finally the problem is formulated as a
constrained non-linear optimisation problem. By solving the optimization problem,
a new CAT02 matrix was derived. With this new matrix, it was found that the
mathematical failure problem of the CIECAM02 is overcome for all the illuminants
considered. Besides, they also found that if the CAT02 with the HPE matrix, the
mathematical failure problem is also overcome for any illuminant. More important,
the HPE matrix makes the CIECAM02 simpler. All the new matrices are under the
evaluation of the CIE TC8-11.

[Link] CIECAM02 Domain is Smaller than that of ICC Profile

Connection Space

The ICC has developed and refined a comprehensive and rigorous system for colour
management [52]. In an ICC colour management work flow, an input colour is
mapped from a device colour space into a colorimetric description for specific
viewing conditions (called the profile connection space—PCS). The PCS is selected
as either CIE XYZ or Lab space under illuminant D50 and the 2◦ observer.
Generally speaking, the input and output devices have different gamuts and, hence,
a gamut mapping is involved. Gamut mapping in XYZ space can cause problems
because of the perceptual non-uniformity of that colour space. Lab space is not
a good space for gamut mapping since lines of constant hue are not generally
straight lines, especially in the blue region [53]. CIECAM02 has been shown to
have a superior perceptual uniformity as well as better hue constancy [40]. Thus,
the CIECAM02 space has been selected as the gamut mapping space.
However, the ICC PCS can contain non-physical colours, which cause problems
when transforming to CIECAM02 space, for example, in the Lightness function J
defined above and the calculation of the parameter defined by
2 CIECAM02 and Its Recent Developments 47

(50000/13)Nc Ncb et (a2 + b2)1/2

t= .
Ra + Ga + (21/20)Ba
When computing J, the value of A can be negative and when computing t, Ra + Ga +
(21/20)Ba can be or near zero. One approach [41, 42] to solving these problems is
to find the domain of CIECAM02 and to pre-clip or map colour values outside of
this domain to fall inside or on this domain boundary, and then the CIECAM02
model can be applied without any problems. The drawbacks of this approach are
that a two step transformation is not easily reversible to form a round trip solution
and clipping in some other colour space would seem to defeat much of the purpose
of choosing CIECAM02 as the gamut mapping space. Another approach [54] is
to extend CIECAM02 so that it will not affect colours within its normal domain
but it will still work, in the sense of being mathematically well defined, for colours
outside its normal domain. To investigate this, the J function and the non-linear post-
adaptation functions in the CIECAM02 were extended. Furthermore, scaling factors
were introduced to avoid the difficulty in calculating the t value. Simulation results
showed this extension of CIECAM02 works very well and full details can be found
in the reference [54]. This approach is also under the evaluation of the CIE TC8-11.

[Link] The HPE Matrix;

Kuo et al. [55] found that the sum of the first row of the HPE matrix (eq. (12)) is
different from unity, which causes a non-zero value of a and b when transforming the
test light source to the reference (equal-energy) light source under full adaptation.
Hence, a slight change to the matrix should be made. For example, the top right
element −0.07868 could be changed to −0.07869. In fact, Kuo et al. [55] suggested
changing each element in the first row slightly.

[Link] The Brightness Function

The brightness function of CIECAM02 is different from the brightness function of

the older CIECAM97s model. The major reason for the change [56] was because of
the correction to the saturation function (s). However, it has been reported that the
brightness prediction of CIECAM02 does not correlate well with the appropriate
visual data [57]. More visual brightness data is needed to clarify the brightness
function.

2.7 Conclusion

This chapter describes the CIECAM02 in great details. Furthermore, more recent
works have been introduced to extend its functions. Efforts were made to reduce the
problems such as mathematical failure for the computation of the lightness attribute.
48 M.R. Luo and C. Li

Overall, the CIECAM02 is capable of accurately predicting colour appearance

under a wide range of viewing conditions. It has been proved to achieve successfully
cross-media colour reproduction (e.g., the reproduction of an image on a display,
on a projection screen or as hard copy) and is adopted by the Microsoft Company
in their latest colour management system, window color system (WCS). With the
addition of CAM02-UCS uniform colour space, size effect and unrelated colours,
it will become a comprehensive colour appearance models to serve most of the
applications.

Appendix: CIE Colour Appearance Model: CIECAM02

Part 1: The Forward Mode

Input: X, Y , Z ( under test illuminant Xw , Yw , Zw )

Output: Correlates of lightness J, chroma C, hue composition H, hue angle h,
colourfulness M, saturation s and brightness Q
Illuminants, viewing surrounds set up and background parameters
(See the note at the end of this Appendix for determining all parameters)
Adopted white in test illuminant: Xw , Yw , Zw
Background in test conditions: Yb
(Reference white in reference illuminant: Xwr = Ywr = Zwr = 100, which are fixed
in the model)
Luminance of test-adapting field (cd/m2 ) : LA
All surround parameters are given in Table 2.3 below
Note that for determining the surround conditions, see the note at the end of this
Appendix. Nc and F are modelled as a function of c, and can be linearly interpolated
as shown in the Fig. 2.16 below, using the above points
Step 0: Calculate all values/parameters which are independent of input samples
⎛ ⎞ ⎛ ⎞
Rw Xw     −L −42  
⎝ Gw ⎠ = MCAT02 · ⎝ Yw ⎠ , 1 A
D = F · 1− · e 92 .
3.6
Bw Zw

Note if D is greater than one or less than zero, set it to one or zero,
respectively.

Yw Yw Yw
DR = D · + 1 − D, DG = D · + 1 − D, DB = D · + 1 − D,
Rw Gw Bw
FL = 0.2 k4 · (5LA ) + 0.1(1 − k4)2 · (5LA )1/3 ,
2 CIECAM02 and Its Recent Developments 49

Table 2.3 Surround F c Nc

parameters
Average 1.0 0.69 1.0
Dim 0.9 0.59 0.9
Dark 0.8 0.535 0.8

Fig. 2.16 Nc and F varies

with c

where k = 1
5·LA +1 .

 0.2
Yb √ 1
n= , z = 1.48 + n, Nbb = 0.725 · , Ncb = Nbb ,
Yw n

⎛ ⎞ ⎛ ⎞ ⎛  ⎞ ⎛ ⎞
Rwc DR · R w Rw Rwc
⎝ Gwc ⎠ = ⎝ DG · Gw ⎠ , ⎝ Gw ⎠ = MHPE · M −1 · ⎝ Gwc ⎠ ,
CAT02
Bwc DB · B w Bw Bwc
⎛ ⎞
0.7328 0.4296 −0.1624
MCAT02 = ⎝ −0.7036 1.6975 0.0061 ⎠ ,
0.0030 0.0136 0.9834
⎛ ⎞
0.38971 0.68898 − 0.07868
MHPE = ⎝ −0.22981 1.18340 0.04641 ⎠ ,
0.00000 0.00000 1.00000
⎛  
0.42 ⎞
FL ·Rw
⎜ 100 ⎟
Raw = 400 · ⎝  0.42 ⎠ + 0.1,

FL ·Rw
100 + 27.13
50 M.R. Luo and C. Li

⎛   ⎞
FL ·Gw 0.42
⎜ 100 ⎟
Gaw = 400 · ⎝   ⎠ + 0.1,
FL ·Gw 0.42
100 + 27.13
⎛   ⎞
FL ·Bw 0.42
⎜ 100 ⎟
Baw = 400 · ⎝   ⎠ + 0.1,
FL ·Bw 0.42
100 + 27.13
 
  Baw
Aw = 2 · Raw + Gaw + − 0.305 · Nbb .
20

Note that all parameters computed in this step are needed for the following
calculations. However, they depend only on surround and viewing condi-
tions; hence, when processing pixels of image, they are computed once for
all. The following computing steps are sample dependent.
Step 1: Calculate (sharpened) cone responses (transfer colour-matching functions
to sharper sensors)
⎛ ⎞ ⎛ ⎞
R X
⎝ G ⎠ = MCAT02 · ⎝ Y ⎠ ,
B Z

Step 2: Calculate the corresponding (sharpened) cone response (considering vari-

ous luminance level and surround conditions included in D; hence, in DR ,
DG and DB ) ⎛ ⎞ ⎛ ⎞
Rc DR · R
⎝ Gc ⎠ = ⎝ DG · G ⎠ ,
Bc DB · B
Step 3: Calculate the Hunt-Pointer-Estevez response
⎛ ⎞ ⎛ ⎞
R Rc
⎝ G ⎠ = MHPE · M −1 · ⎝ Gc ⎠ ,
CAT02
B Bc

Step 4: Calculate the post-adaptation cone response (resulting in dynamic range

compression)
⎛  0.42 ⎞
FL ·R
⎜ 100 ⎟
Ra = 400 · ⎝   ⎠ + 0.1.
FL ·R  0.42
100 + 27.13

If R is negative, then
2 CIECAM02 and Its Recent Developments 51

Table 2.4 Unique hue data Red Yellow Green Blue Red
for calculation of hue
quadrature i 1 2 3 4 5
hi 20.14 90.00 164.25 237.53 380.14
ei 0.8 0.7 1.0 1.2 0.8
Hi 0.0 100.0 200.0 300.0 400.0

⎛  0.42 ⎞
−FL ·R
⎜ 100 ⎟
Ra = −400 · ⎝  0.42 ⎠ + 0.1
−FL ·R 
100 + 27.13

and similarly for the computations of Ga , and Ba , respectively.

Step 5: Calculate Redness–Greenness (a) , Yellowness–Blueness (b) components
and hue angle (h):

12 · Ga Ba
a = Ra − + ,
11 11
(R + Ga − 2 · Ba)
 
b= a ,
9
 
b
h = tan−1
a

make sure h between 0 and 360◦.

Step 6: Calculate eccentricity (et ) and hue composition (H), using the unique hue
data given in Table 2.4; set h = h + 360 if h < h1 , otherwise h = h. Choose
a proper i(i =1,2,3 or 4) so that hi ≤ h < hi+1 . Calculate
    
1 h ·π
et = · cos + 2 + 3.8 ,
4 180
which is close to, but not exactly the same as, the eccentricity factor given
in Table 2.4.

100 · h −h
ei
i
H = Hi + h −hi hi+1 −h
.
ei + ei+1

Step 7: Calculate achromatic response A

 
B
A = 2 · Ra + Ga + a − 0.305 · Nbb .
20
Step 8: Calculate the correlate of lightness
 c·z
A
J = 100 · .
Aw
52 M.R. Luo and C. Li

Step 9: Calculate the correlate of brightness

   
4 J 0.5
Q= · · (Aw + 4) · FL0.25 .
c 100

Step 10: Calculate the correlates of chroma (C), colourfulness (M) and
saturation (s)
 50000   1/2
· Nc · Ncb · et · a2 + b2
t= 13
  ,
Ra + Ga + 21 
20 · Ba
 
J 0.5
C = t 0.9 · · (1.64 − 0.29n)0.73 ,
100
M = C · FL0.25 ,
 0.5
M
s = 100 · .
Q

Part 2: The Reverse Mode

Input: J or Q; C, M or s; H or h
Output: X,Y, Z ( under test illuminant Xw ,Yw , Zw )
Illuminants, viewing surrounds and background parameters are the same as
those given in the forward mode. See notes at the end of this Appendix calculat-
ing/defining the luminance of the adapting field and surround conditions.
Step 0: Calculate viewing parameters
Compute all FL , n, z, Nbb = Nbc , Rw , Gw , Bw , D, DR , DG , DB , Rwc , Gwc , Bwc ,
Rw , Gw , Bw Raw , Gaw , Baw and Aw using the same formulae as in Step 0 of
the Forward model. They are needed in the following steps. Note that all
data computed in this step can be used for all samples (e.g., all pixels for an
image) under the viewing conditions. Hence, they are computed once for
all. The following computing steps are sample dependent.
Step 1: Obtain J, C and h from H, Q, M, s
The entering data can be in different combination of perceived correlates,
i.e., J or Q; C, M, or s; and H or h. Hence, the followings are needed to
convert the others to J, C, and h.
Step 1–1: Compute J from Q (if start from Q)
 2
c·Q
J = 6.25 · .
(Aw + 4) · FL0.25
2 CIECAM02 and Its Recent Developments 53

Step 1–2: Calculate C from M or s

M
C= (if start from M)
FL0.25
   
4 J 0.5
Q= · · (Aw +4.0) · FL0.25
c 100
 s 2
 Q
and C = 100 · ( F 0.25 ) (if start from s)
L
Step 1–3: Calculate h from H (if start from H)
The correlate of hue (h) can be computed by using data in Table 2.4 in
the Forward mode.
Choose a proper i (i = 1,2,3 or 4) so that Hi ≤ H < Hi+1 .

(H − Hi ) · (ei+1 hi − ei · hi+1 ) − 100 · hi · ei+1

h = .
(H − Hi ) · (ei+1 − ei ) − 100 · ei+1

Set h = h − 360 if h > 360, otherwise h = h .

Step 2: Calculate t, et , p1 , p2 and p3
⎡ ⎤ 1
0.9
C
t = ⎣ ⎦ ,
J
100 · (1.64 − 0.29n)0.73
1   π  
et = · cos h · + 2 + 3.8 ,
4 180
 1
J c·z
A = Aw · ,
100
   
50000 1
p1 = · Nc · Ncb · et · , if t = 0,
13 t
A
p2 = + 0.305,
Nbb
21
p3 = ,
20

Step 3: Calculate a and b

If t = 0, then a = b = 0 and go to Step 4
(be sure transferring h from degree to radian before calculating sin(h) and
cos(h))
If | sin(h)| ≥ | cos(h)|, then
54 M.R. Luo and C. Li

p1
p4 = ,
sin(h)
 460 
p2 · (2 + p3) · 1403
b=  220   cos(h)   27   ,
p4 + (2 + p3) · 1403 · sin(h) − 1403 + p3 · 6300
1403
 
cos(h)
a = b· .
sin(h)

If | cos(h)| > | sin(h)|, then

p1
p5 = ,
cos(h)
 460 
p2 · (2 + p3) · 1403
a=  220   27     sin(h)  ,
p5 + (2 + p3) · 1403 − 1403 − p3 · 6300 1403 · cos(h)
 
sin(h)
b = a· .
cos(h)

Step 4: Calculate Ra , Ga and Ba

460 451 288

Ra = · p2 + ·a+ · b,
1403 1403 1403
460 891 261
Ga = · p2 − ·a− · b,
1403 1403 1403
460 220 6300
Ba = · p2 − ·a− · b.
1403 1403 1403

Step 5: Calculate R , G and B

  1
 100 27.13 · |Ra − 0.1| 0.42
R = sign(Ra − 0.1) · · .
FL 400 − |Ra − 0.1|
⎧
⎨ 1 if x > 0
Here, sign(x) = 0 if x = 0 , and similarly computing G , and B from
⎩
−1 if x < 0
Ga , and Ba .
Step 6: Calculate RC , GC and BC (for the inverse matrix, see the note at the end of
the Appendix)
⎛ ⎞ ⎛ ⎞
Rc R
⎝ Gc ⎠ = MCAT02 · M −1 ⎝ G ⎠ .
HPE ·
Bc B
2 CIECAM02 and Its Recent Developments 55

Step 7: Calculate R, G and B

⎛ ⎞ ⎛ Rc ⎞
R DR
⎝G⎠ = ⎜ G ⎟
⎝ DGc ⎠ .
B Bc
D B

Step 8: Calculate X, Y and Z (for the coefficients of the inverse matrix, see the note
at the end of the Appendix)
⎛ ⎞ ⎛ ⎞
X R
⎝ Y ⎠ = M −1 · ⎝ G ⎠ .
CAT02
Z B

Notes to Appendix

1. It is recommended to use the matrix coefficients given below for the inverse
−1 −1
matrix MCAT02 and MHPE :
⎛ ⎞
1.096124 −0.278869 0.182745
−1
MCAT02 = ⎝ 0.454369 0.473533 0.072098 ⎠ ,
−0.009628 −0.005698 1.015326
⎛ ⎞
1.910197 −1.112124 0.201908
M −1 = ⎝ 0.370950 0.629054 −0.000008 ⎠
HPE
0.000000 0.000000 1.000000

2. For implementing the CIECAM02, the testing data and the corresponding results
from the forward and reverse modes can be found from reference 7.
3. The LA is computed using (2.11)
   
EW Yb LW ·Yb
LA = · = , (2.11)
π YW YW
where Ew = π ·Lw is the illuminance of reference white in lux unit; Lw the luminance
of reference white in cd/m2 unit, Yb the luminance factor of the background and Yw
the luminance factor of the reference white.

References

1. Luo MR (1999) Colour science: past, present and future. In: MacDonald LW and Luo MR
(Eds) Colour imaging: vision and technology. Wiley, New York, 384–404
2. CIE Technical Report (2004) Colorimetry, 3rd ed. Publication 15:2004, CIE Central Bureau,
Vienna.
56 M.R. Luo and C. Li

3. Luo MR, Cui GH, Rigg B (2001) The development of the CIE 2000 colour difference formula.
Color Res Appl 26:340-350.
4. Luo MR, Hunt RWG (1998) The structure of the CIE 1997 colour appearance model
(CIECAM97s). Color Res Appl 23:138–146
5. CIE (1998) The CIE 1997 interim colour appearance model (simple version), CIECAM97s.
CIE Publication 131, CIE Central Bureau, Vienna, Austria.
6. Moroney N, Fairchild MD, Hunt RWG, Li C, Luo MR, Newman T (2002) The CIECAM02
color appearance model, Proceedings of the 10th color imaging conference, IS&T and SID,
Scottsdale, Arizona, 23–27
7. CIE (2004) A colour appearance model for colour management systems: CIECAM02, CIE
Publication 159 CIE Central Bureau, Vienna, Austria
8. Luo MR and Li CJ (2007) CIE colour appearance models and associated colour spaces, Chapter
11 of the book: colorimetry-understanding the CIE System. In: Schanda J (ed) Wiley, New York
9. Luo MR, Cui GH, Li CJ and Rigg B (2006) Uniform colour spaces based on CIECAM02
colour appearance model. Color Res Appl 31:320–330
10. Xiao K, Luo MR, Li C, Hong G (2010) Colour appearance prediction for room colours, Color
Res Appl 35:284–293
11. Xiao K, Luo MR, Li CJ, Cui G, Park D (2011) Investigation of colour size effect for colour
appearance assessment, Color Res Appl 36:201–209
12. Xiao K, Luo MR, Li CJ (2012) Color size effect modelling, Color Res Appl 37:4–12
13. Fu CY, Li CJ, Luo MR, Hunt RWG, Pointer MR (2007) Quantifying colour appearance for
unrelated colour under photopic and mesopic vision, Proceedings of the 15th color imaging
conference, IS&T and SID, Albuquerque, New Mexico, 319–324
14. Li CJ, Chorro-Calderon E, Luo MR, Pointer MR (2009) Recent progress with extensions to
CIECAM02, Proceedings of the 17th color imaging conference, IS&T and SID, Albuquerque,
New Mexico 69–74
15. CIE Publ. 17.4:1987, International lighting vocabulary, the 4th edition
16. Mori L, Sobagaki H, Komatsubara H, Ikeda K (1991) Field trials on CIE chromatic adaptation
formula. Proceedings of the CIE 22nd session, 55–58
17. McCann JJ, McKee SP, Taylor TH (1976) Quantitative studies in Retinex theory: a comparison
between theoretical predictions and observer responses to the ‘color mondrian ’ experiments.
Vision Res 16:445–458
18. Breneman EJ (1987) Corresponding chromaticities for different states of adaptation to complex
visual fields. J Opt Soc Am A 4:1115–1129
19. Helson H, Judd DB, Warren MH (1952) Object-color changes from daylight to incandescent
filament illumination. Illum Eng 47:221–233
20. Lam KM (1985) Metamerism and colour constancy. Ph.D. thesis, University of Bradford, UK
21. Braun KM, Fairchild MD (1996) Psychophysical generation of matching images for cross-
media colour reproduction. Proceedings of 4th color imaging conference, IS&T, Springfield,
Va., 214–220
22. Luo MR, Clarke AA, Rhodes PA, Schappo A, Scrivener SAR, Tait C (1991) Quantifying colour
appearance. Part I. LUTCHI colour appearance data. Color Res Appl 16:166–180
23. Luo MR, Gao XW, Rhodes PA, Xin HJ, Clarke AA, Scrivener SAR (1993) Quantifying colour
appearance, Part IV: transmissive media. Color Res Appl 18:191–209
24. Kuo WG, Luo MR, Bez HE (1995) Various chromatic adaptation transforms tested using new
colour appearance data in textiles. Color Res Appl 20:313–327
25. Juan LY, Luo MR (2000) New magnitude estimation data for evaluating colour appearance
models. Colour and Visual Scales 2000, NPL, 3-5 April, UK
26. Juan LY, Luo MR (2002) Magnitude estimation for scaling saturation. Proceedings of 9th
session of the association internationale de la couleur (AIC Color 2001), Rochester, USA,
(June 2001), Proceedings of SPIE 4421, 575–578
27. Li CJ, Luo MR, Rigg B, Hunt RWG (2002) CMC 2000 chromatic adaptation transform:
CMCCAT2000. Color Res Appl 27:49–58
2 CIECAM02 and Its Recent Developments 57

28. Judd DB (1940), Hue, saturation, and lightness of surface colors with chromatic illumination.
J Opt Soc Am 30:2–32
29. Kries V (1902), Chromatic adaptation, Festschrift der Albrecht-Ludwig-Universitat (Fribourg),
[Translation: MacAdam DL, Sources of Color Science, MIT Press, Cambridge, Mass. (1970)]
30. Luo MR, Hunt RWG (1998) A chromatic adaptation transform and a colour inconstancy index.
Color Res Appl 23:154–158
31. Li CJ, Luo MR, Hunt RWG (2000) A revision of the CIECAM97s Model. Color Res Appl
25:260–266
32. Hunt RWG, Li CJ, Juan LY, Luo MR (2002), Further improvements to CIECAM97s. Color
Res Appl 27:164–170
33. Finlayson GD, Süsstrunk S (2000) Performance of a chromatic adaptation transform based on
spectral sharpening. Proceedings of IS&T/SID 8th color imaging conference, 49–55
34. Hunt RWG (1952) Light and dark adaptation and perception of color. J Opt Soc Am
42:190–199
35. Stevens JC, Stevens SS (1963) Brightness functions: effects of adaptation. J. Opt Soc Am
53:375–385
36. Bartleson CJ, Breneman EJ (1967) Brightness perception in complex fields. J. Opt Soc Am
57:953–957
37. Luo MR, Gao XW, Sciviner SAR (1995) Quantifying colour appearance, Part V, Simultaneous
contrast. Color Res Appl 20:18–28
38. Wyszecki G, Stiles WS (1982) Color Science: concepts and methods, Quantitative data and
formulae. Wiley, New York
39. Helson H (1938) Fundamental problems in color vision. I. The principle governing changes in
hue, saturation, and lightness of non-selective samples in chromatic illumination. J Exp Psych
23:439–477
40. CIE Publ. 152:2003, Moroney N, Han Z (2003) Field trials of the CIECAM02 colour
appearance, Proceedings of the 25th session of the CIE, San Diego D8-2–D8-5.
41. Tastl I, Bhachech M, Moroney N, Holm J (2005) ICC colour management and CIECAM02,
Proceedings of the 13th of CIC, p 318
42. Gury R, Shaw M (2005) Dealing with imaginary color encodings in CIECAM02 in an ICC
workflow. Proceedings of the 13th of CIC, pp 217–223
43. Li CJ, Luo MR, Cui GH (2003) Colour-difference evaluation using colour appearance models.
The 11th Color Imaging Conference, IS&T and SID, Scottsdale, Arizona, November, 127–131
44. Luo MR, Rigg B (1986) Chromaticity–discrimination ellipses for surface colours. Color Res
Appl 11:25–42
45. Berns RS, Alman DH, Reniff L, Snyder GD, Balonon-Rosen MR (1991) Visual determi-
nation of suprathreshold color-difference tolerances using probit analysis. Color Res Appl
16:297–316
46. Hunt RWG (1952) Measuring colour, 3rd edition, Fountain Press, Kingston-upon-Thames,
1998
47. Li CJ, Chorro-Calderon E, Luo MR, Pointer MR (2009) Recent progress with extensión to
CIECAM02, Seventeenth Colour Imaging Conference, Final Program and Proceedings, 69–74
48. Brill MH (2006) Irregularity in CIECAM02 and its avoidance. Color Res Appl 31(2):142–145
49. Brill MH, Susstrunk S (2008) Repairing gamut problems in CIECAM02: a progress report.
Color Res Appl 33(5):424–426
50. Süsstrunk S, Brill M (2006) The nesting instinct: repairing non nested gamuts in CIECAM02.
14th SID/IS&T color imaging conference
51. Li CJ, Perales E, Luo MR, Martı́nez-Verdú F, A Mathematical approach for predicting non-
negative tristimulus values using the CAT02 chromatic adaptation transform, Color Res Appl
(in press)
52. ISO 15076-1 (2005) Image technology, colour management-Architecture, profile format and
data structure-Part I: based on ICC.1:2004-10, [Link]
53. Moroney N (2003) A hypothesis regarding the poor blue constancy of CIELAB. Color Res
Appl 28(5):371–378
58 M.R. Luo and C. Li

54. W. Gill GW (2008) A solution to CIECAM02 numerical and range issues, Proceedings of the
16th of color imaging conference, IS&T and SID, Portland, Oregan 322–327
55. Kuo CH, Zeise E, Lai D (2006) Robust CIECAM02 implementation and numerical experiment
within an ICC workflow. Proceedings of the 14th of CIC, pp 215–219
56. Hunt RWG, Li CJ, Luo MR (2002) Dynamic cone response functions for modes of colour
appearance. Color Res Appl 28:82–88
57. Paula J, Alessi P (2008) Private communication pursuit of scales corresponding to equal
perceptual brightness, personal correspondence
Chapter 3
Colour Difference Evaluation

Manuel Melgosa, Alain Trémeau, and Guihua Cui

In the black, all the colors agree

Francis Bacon

Abstract For a pair of homogeneous colour samples or two complex images

viewed under specific conditions, colour-difference formulas try to predict the visu-
ally perceived (subjective) colour difference starting from instrumental (objective)
colour measurements. The history related to the five up-to-date CIE-recommended
colour-difference formulas is reviewed, with special emphasis on the structure and
performance of the last one, CIEDE2000. Advanced colour-difference formulas
with an associated colour space (e.g., DIN99d, CAM02, Euclidean OSA-UCS, etc.)
are also discussed. Different indices proposed to measure the performance of a given
colour-difference formula (e.g., PF/3, STRESS, etc.) are reviewed. Among current
trends on colour-difference evaluation, it can be mentioned the research activities
carried out by different CIE Technical Committees (e.g., CIE TC’s 1-55, 1-57, 1-63,
1-81 and 8-02), the need of new reliable experimental datasets, the development
of colour-difference formulas based on IPT and colour-appearance models, and
the concept of “total differences,” which considers the interactions between colour
properties and other object attributes like texture, translucency, and gloss.

M. Melgosa ()
Departamento de Optica, Facultad de Ciencias, Universidad de Granada, Spain
e-mail: mmelgosa@[Link]
A. Trémeau
Laboratory Hubert Curien, UMR CNRS 5516, Jean Monnet University, Saint-Etienne, France
e-mail: [Link]@[Link]
G. Cui
VeriVide Limited, Leicester, LE19 4SG, United Kingdom
e-mail: [Link]@[Link]

C. Fernandez-Maloigne (ed.), Advanced Color Image Processing and Analysis, 59

DOI 10.1007/978-1-4419-6190-7 3,
© Springer Science+Business Media New York 2013
60 M. Melgosa et al.

Keywords Colour-difference formula • Uniform colour space • CIELUV •

CIELAB • CIE94 • CIEDE2000 • DIN99 • CAM02-SCD • CAM02-LCD •
CAM02-UCS • S-CIELAB • IPT • PF/3 • STRESS

3.1 Introduction

From two homogeneous colour stimuli, we can ask ourselves what is the magnitude
of the perceived colour difference between them. Of course, this question may
also be asked in the case of more complex stimuli like two colour images.
In fact, to achieve a consistent answer to the previous question, we must first
specify the experimental observation conditions: for example, size of the stimuli,
background behind them, illuminance level, etc. It is well known that experimental
illuminating and viewing conditions (the so-called “parametric effects”) play an
important role on the magnitude of perceived colour differences, as reported by
the International Commission on Illumination (CIE) [1]. Specifically, to avoid the
spread of experimental results under many different observation conditions, in 1995
the CIE proposed [2] to analyze just 17 “colour centers” well distributed in colour
space (Table 3.1), under a given set of visual conditions similar to those usually
found in industrial practice, which are designated as “reference conditions,” and are
as follows:
Illumination: D65 source
Illuminance: 1000 lx
Observer: Normal colour vision
Background field: Uniform, neutral grey with L∗ = 50

Table 3.1 The 17 colour Name L∗10 a∗10 b∗10

centers proposed by CIE for
further coordinated research 1. Grey 62 0 0
on colour-difference 2. Red 44 37 23
evaluation [2]. Bold letters 3. Red, high chroma 44 58 36
indicate five colour centers 4. Orange 63 13 21
used as experimental controls, 5. Orange, high chroma 63 36 63
which were earlier proposed 6. Yellow 87 −7 47
with the same goal by CIE 7. Yellow, high chroma 87 −11 76
(A.R. Robertson, Color Res. 8. Yellow-green 65 −10 13
Appl. 3, 149–151, 1978)
9. Yellow-green, high chroma 65 −30 39
10. Green 56 −32 0
11. Green, high chroma 56 −45 0
12. Blue-green 50 −16 −11
13. Blue-green, high chroma 50 −32 −22
14. Blue 36 5 −31
15. Blue, high chroma 34 7 −44
16. Purple 46 12 −13
17. Purple, high chroma 46 26 −26
3 Colour Difference Evaluation 61

Viewing mode: Object

Sample size: Greater than four degrees
Sample separation: Direct edge contact
∗
Sample colour-difference magnitude: Lower than 5.0 ΔEab
Sample structure: Homogeneous (without texture)
The perceived visual difference between two colour stimuli is often designated
as ΔV , and it is just the subjective answer provided by our visual system. It
must be mentioned that large inter- and intra-observer variability (sometimes
designated as accuracy and repeatability, respectively) can be found determining
visual colour differences, even in carefully designed experiments on colour dif-
ference evaluation [3]. Intra and inter-observer variability was rarely considered
in old experiments (e.g., the pioneer MacAdam’s experiment [4] producing x, y
chromaticity discrimination ellipses involved just one observer), but it is essential in
modern experiments on colour differences [5], because individuals give results that
rarely correlate with those of population. Although different methods have been
proposed to obtain the visual difference ΔV in a colour pair, the two most popular
ones are the “anchor pair” and “grey scale” methods. In “anchor pair” experiments
[6], the observer just reports whether the colour difference in the colour pair is
smaller or greater than the one shown in a fixed neutral colour pair. In “grey scale”
experiments [7], the observer compares the colour difference in the test pair with
a given set of neutral colour pairs with increasing colour-difference magnitudes,
choosing the one with the closest colour difference to the test pair (Fig. 3.1).
Commercial grey-scales with colour-difference pairs in geometrical progression are
currently available, [8–10] although the most appropriate grey scale to be used in
experiments is now questioned [11]. It has been reported [12] that “anchor pair” and
“grey scale” experiments conduct only to qualitative analogous results.
In many industrial applications, it is highly desirable to predict the subjective
visual colour difference ΔV from objective numerical colour specifications; specifi-
cally, the tristimulus values measurements of the two samples in a colour pair. This is
just the main goal of the so-called “colour-difference formulas”. A colour-difference
formula can be defined as a computation providing a non-negative value ΔE from
the tristimulus values of the two samples in a colour pair. It is worth to mention that
in modern colour-difference formulas, additional information on parameters related
to observation conditions is also considered to compute ΔE:

ΔE = f (X1 ,Y1 , Z1 , X2 ,Y2 , Z2 , Observation Conditions Parameters) (3.1)

While ΔV is the result of a subjective measurement like the average of the visual
assessments performed by a panel of observers, using a specific method and working
under fixed observation conditions, ΔE is an objective measurement which can be
currently performed using colorimetric instrumentation. Obviously, the main goal is
to achieve a ΔE analogous to ΔV for any colour pair in colour space and under any
visual set of observational conditions. In this way, complex tasks like visual pass/fail
decisions in a production chain could be done in a completely automatic way
62 M. Melgosa et al.

Fig. 3.1 A yellow colour pair of textile samples, together with a grey scale for visual assessment
of the colour difference in such a pair. A colour mask may be employed to choose a colour pair
in the grey scale, or to have a test pair with the same size than those in the grey scale. Photo from
Dr. Michal Vik, Technical University of Liberec, Czech Republic

Fig. 3.2 Visual versus instrumental color-difference evaluation: example of quality control using
a colorimeter. Photo from “Precise Color Communication”, Konica-Minolta Sensing, Inc., 1998

(Fig. 3.2). However, it must be recognized that this is a very ambitious goal, because
in fact it is intended to predict the final answer of our visual system, currently
unknown in many aspects. Anyway, important advances have been produced in
colour-difference measurement, as will be described in the next section.
3 Colour Difference Evaluation 63

Three different steps can be distinguished in the history of modern colorimetry:

colour matching, colour differences, and colour appearance. Colour matching
culminates with the definition of the tristimulus values X,Y, Z: Two stimuli, viewed
under identical conditions, match for a specific standard observer when their
tristimulus values are equal. This defines “basic colorimetry” and is the basis for
numerical colour specification [13]. However, when tristimulus values are unequal,
the match may not persist, depending on the magnitude of dissimilarity (i.e., larger
or not than a threshold difference). Tristimulus values X,Y, Z (or x, y,Y coordinates)
should never be used as direct estimates of colour differences. Colour-difference
formulas can be used to measure the dissimilarity between two colour stimuli of the
same size and shape, which are observed under the same visual conditions. Relating
numerical differences to perceived colour differences is one of the challenges of
so-called “advanced colorimetry”. Finally, colour appearance is concerned with the
description of what colour stimuli look like under a variety of visual conditions.
More specifically, a colour appearance model provides a viewing condition-specific
method for transforming tristimulus values to and/or from perceptual attribute
correlates [14]. Application of such models open up a world of possibilities for
the accurate specification, control and reproduction of colour, and may eventually
include in the future the field of colour differences [15].

3.2 The CIE Recommended Colour-Difference Formulas

As described by Luo [16], first colour-difference formulas were based on the

Munsell system, followed by formulas based on MacAdam’s data, and linear
(or non-linear) transformations of tristimulus values X,Y, Z. The interested reader
can find useful information in the literature [17–19] about many colour-difference
formulas proposed in the past. Colour-difference formulas have been considered in
CIE programs since the 1950s, and in this section we will focus on the five up-to-
date CIE-recommended colour-difference formulas.
The first CIE-recommended colour-difference formula was proposed in 1964 as
the Euclidean distance in the CIE U∗ , V∗ , W∗ colour space. This space was actually
based on MacAdam’s 1960 uniform colour scales (CIE 1960 UCS), which intended
to improve the uniformity of the CIE 1931 x, y chromaticity diagram. In 1963,
Wyszecki added the third dimension to this space. The currently proposed CIE
Colour Rendering Index [20] is based on the CIE 1964 U∗ , V∗ , W∗ colour-difference
formula.
A landmark was achieved in 1976 with the joint CIE recommendation of the
CIELUV and CIELAB colour spaces and colour-difference formulas [21]. As
described by Robertson [22] in 1976 the CIE recommended the use of two approx-
imately uniform colour spaces and colour-difference formulas, which were chosen
from among several of similar merit to promote uniformity of practice, pending
the development of a space and formula giving substantially better correlation
64 M. Melgosa et al.

∗
with visual judgments. While the CIELAB colour-difference formula ΔEab had the
advantage that it was very similar to the Adams–Nickerson (ANLAB40) formula, al-
ready adopted by several national industrial groups, the CIELUV colour-difference
∗
formula ΔEuv had the advantage of a linear chromaticity diagram, particularly
useful in lighting applications. It can be said that the CIELAB colour-difference
formula was soon accepted by industry: while in 1977 more than 20 different
colour-difference formulas were employed in the USA industry, 92% of these
industries had adopted CIELAB in 1992 [23]. Because there are no fixed scale
factors between the results provided by two different colour-difference formulas,
the uniformity of practice (standardization) achieved by CIELAB was an important
achievement for industrial practice. It should be said that a colour difference
between 0.4 and 0.7 CIELAB units is approximately a just noticeable or threshold
difference, although even lower values of colour differences are sometimes managed
by specific industries. Colour differences between contiguous samples in colour
atlases (e.g., Munsell Book of Color) are usually greater than 5.0 CIELAB units,
being designated as large colour differences.
After the proposal of CIELAB, many CIELAB-based colour-difference for-
mulas were proposed with considerable satisfactory results [24]. Among these
CIELAB-based formulas, it is worth mentioning the CMC [25] and BFD [26]
colour-difference formulas. The CMC formula was recommended by the Colour
Measurement Committee of the Society of Dyers and Colourists (UK), and inte-
grated into some ISO standards. CIELAB lightness, chroma, and hue differences
are properly weighted in the CMC formula, which also includes parametric factors
dependent on visual conditions (e.g., the CMC lightness differences have half value
for textile samples). In 1995 the CIE proposed the CIE94 colour-difference formula
[27], which may be considered a simplified version of CMC. CIE94 was based
on most robust trends in three reliable experimental datasets, proposing simple
corrections to CIELAB (linear weighting functions of the average chroma for the
CIELAB chroma and hue differences), as well as parametric factors equal to 1.0
under the so-called “reference conditions” (see Introduction). It can be said that
CIE94 adopted a versatile but too conservative approach adopting only the most
well-known CIELAB corrections, like the old chroma-difference correction already
suggested by McDonald for the ANLAB formula in 1974 [28].
In 2001 the CIE recommended its last colour-difference formula, CIEDE2000
[29]. From a combined dataset of reliable experimental data containing 3,657 colour
pairs from four different laboratories, the CIEDE2000 formula was developed [30].
The CIEDE2000 formula has the same final structure as the BFD [26] formula.
Five corrections to CIELAB were included in CIEDE2000: A weighting function
for lightness accounting for the “crispening effect” produced by an achromatic
background with lightness L∗ = 50; a weighting function for chroma identical to
the one adopted by the previous CIE94 formula; a weighting function for hue which
is dependent on both hue and chroma; a correction of the a∗ coordinate for neutral
colours; and a rotation term which takes account of the experimental chroma and
hue interaction in the blue region. The most important correction to CIELAB in
CIEDE2000 is the chroma correction [31]. CIEDE2000 also includes parametric
3 Colour Difference Evaluation 65

factors with values kL = kC = kH = 1 under the “reference conditions” adopted

by CIE94 and mentioned in the previous section. Starting from CIELAB, the
mathematical equations defining the CIEDE2000 [29] colour-difference formula,
noted ΔE00 , are as follows:
 2  2  2   
ΔL ΔC ΔH  ΔC ΔH 
ΔE00 = + + + RT (3.2)
kL S L kC SC kH S H kC SC kH S H

First, for each one of the two colour samples, designated as “b” (“batch”) and “s”
(“standard”), a localized modification of the CIELAB coordinate a∗ is made:

L = L∗ (3.3)
 ∗
a = (1 + G) a (3.4)
b = b∗ (3.5)
⎛  ⎞

 ∗7
Cab
G = 0.5 ⎝1 − ⎠ (3.6)
∗7 + 257
Cab

where the upper bar means arithmetical mean of standard and batch. Transformed
a , b are used in calculations of transformed chroma and hue angle, in the usual
way [21]:
!
C  = a 2 + b 2 (3.7)
 
h = arctan ba (3.8)

Lightness-, chroma-, and hue-differences employed in (3.2) are computed as

follows:

ΔL = Lb − Ls (3.9)

ΔC = Cb − Cs (3.10)

Δh = hb − hs (3.11)
 
Δh
ΔH  = 2 Cb Cs sin (3.12)
2
The “weighting functions” for lightness, chroma, and hue, where once again the
upper bars means arithmetical mean of standard and batch, are as follows:
 2
0.015 L − 50
SL = 1 +  2 (3.13)
20 + L − 50

SC = 1 + 0.045C (3.14)
66 M. Melgosa et al.

SH = 1 + 0.015 C T (3.15)
   
T = 1 − 0.17 cos h − 30◦ + 0.24 cos 2h
   
+0.32 cos 3h + 6◦ − 0.20 cos 4h − 63◦ (3.16)

Finally, the rotation term RT is defined by the next equations:

RT = − sin (2Δθ ) RC (3.17)

"   2 #
Δθ = 30 exp − h − 275◦ /25 (3.18)


 C 7
RC = 2 (3.19)
C + 257
7

Statistical analyses confirmed that CIEDE2000 significantly improved both

CIE94 and CMC colour-difference formulas, for the experimental combined dataset
employed at its development [30], and therefore it was proposed to the scientific
community. Figure 3.3 shows that experimental colour discrimination ellipses [30]
in CIELAB a∗ b∗ plane are in very good agreement with predictions made by the
CIEDE2000 colour-difference formula.
Sharma et al. [32] have pointed out different problems in the computation
of CIEDE2000 colour differences, which were not detected at its development.

Specifically, these problems come from Δh and h values when samples are placed
in different angular sectors, which leads to discontinuities in T (3.16) and Δθ
(3.18) values. In the worst case, these discontinuities produced a deviation of
0.27 CIEDE2000 units for colour differences up to 5.0 CIELAB units, and were
∗
around 1% for threshold (ΔEab < 1.0) colour differences, which can be considered
negligible in most cases.
Currently CIEDE2000 is the CIE-recommended colour-difference formula, and
CIE TC 1–57 “Standards in Colorimetry” is now in the way to propose this formula
as a CIE standard. Anyway, CIEDE2000 cannot be considered a final answer to the
problem of colour difference evaluation [33]. At this point, it is very interesting
to note that CIEDE2000 (and also CIE94) are CIELAB-based colour-difference
formulas which have not an associated colour space, as it should be desirable and
discussed in the next section.

3.3 Advanced Colour-Difference Formulas

Under this epigraph, we are going to mention some recent colour-difference

formulas with an associated colour space; that is, alternative colour spaces to
CIELAB where the simple Euclidean distance between two points provides the
corresponding colour difference.
3 Colour Difference Evaluation 67

120
b*

90

60

30

a*
0

−30

−60
−60 −30 0 30 60 90 120

Fig. 3.3 Experimental colour discrimination ellipses in CIELAB a∗ b∗ for the BFD and
RIT-DuPont datasets (red), compared with predictions made by the CIEDE2000 colour-difference
formula (black) [30]

In 1999, K. Witt proposed the DIN99 colour-difference formula (Witt, 1999,

DIN99 colour-difference formula, a Euclidean model, Private communication), later
adopted as the German standard DIN6176 [34]. The DIN99 colour space applies a
logarithmic transformation on the CIELAB lightness L∗ , and a rotation and stretch
on the chroma plane a∗ b∗ , followed by a chroma compression inspired in the CIE94
weighting function for chroma. The DIN99 colour-difference formula is just the
Euclidean distance in the DIN99 colour space. In 2002, Cui et al. [35] proposed
different uniform colour spaces based on DIN99, the DIN99d being the one with
the best performance. In DIN99d space the tristimulus value X was modified
by subtracting a portion of Z to improve the performance in the blue region,
as suggested by Kuehni [36]. Equations defining the DIN99d colour-difference
formula, noted as ΔE99d , are as follows:

ΔE99d = ΔL299d + Δa299d + Δb299d (3.20)

where the symbol “Δ” indicates differences between batch and standard samples in
the colour pair. For each one of the two samples (and also the reference white), the
next equations based on CIELAB L∗ , a∗ , b∗ coordinates are applied:
68 M. Melgosa et al.

X  = 1.12X − 0.12Z (3.21)

L99d = 325.22 ln (1 + 0.0036L∗) (3.22)
∗ ◦ ∗ ◦
e = a cos (50 ) + b sin (50 ) (3.23)
f = 1.14 [−a∗ sin (50◦) + b∗ cos (50◦ )] (3.24)

where the new e and f coordinates are the result of a rotation and re-scaling of the
CIELAB a∗ b∗ coordinates, and L99d is not too different to CIELAB lightness L∗ .
!
G= e2 + f 2 (3.25)
C99d = 22.5 ln (1 + 0.06G) (3.26)

∗ . Finally:
where this new chroma C99d is a compression of CIELAB chroma Cab

h99d = arctan ( f /e) + 50◦ (3.27)

a99d = C99d cos (h99d ) (3.28)
b99d = C99d sin (h99d ) (3.29)

In 2006, on the basis of the CIECAM02 colour appearance model [14], three new
Euclidean colour-difference formulas were proposed [37] for small (CAM02-SCD),
large (CAM02-LCD), and all colour differences (CAM02-UCS). In these CAM02
formulas a non-linear transformation to CIECAM02 lightness J, and a logarithmic
compression to the CIECAM02 colourfulness M were applied. The corresponding
equations are as follows:

 $ 2
ΔECAM02 = Δ J  KL + (Δa )2 + (Δb )2 (3.30)
(1 + 100 c1) J
J = (3.31)
1 + c1 J
M  = (1/c2 ) ln (1 + c2M) (3.32)
a = M  cos (h) (3.33)
 
b = M sin (h) (3.34)

where J, M, and h are the CIECAM02 lightness, colourfulness, and hue angle values,
respectively. In addition, the ΔJ  , Δa , and Δb are the J  , a , and b differences
between the standard and batch in a colour pair. Finally, the parameter KL has
values 0.77, 1.24, and 1.00 for the CAM02-LCD, CAM02-SCD, and CAM02-UCS
formulas, respectively, while c1 = 0.007 for all these formulas, and c2 has values
0.0053, 0.0363, and 0.0228, for the CAM02-LCD, CAM02-SCD, and CAM02-UCS
formulas, respectively [37]. The results achieved by these CAM02 formulas are very
encouraging: embedded uniform colour space in the CIECAM02 colour appearance
3 Colour Difference Evaluation 69

model can be useful to make successful predictions of colour differences; that is,
colour difference may be a specific aspect of colour appearance. Berns and Xue
have also proposed colour-difference formulas based on the CIECAM02 colour
appearance model [38].
OSA-UCS is a noteworthy empirical colour system for large colour differences
developed in 1974 by the Optical Society of America’s committee on Uniform Color
Scales [39]. In this system, the straight lines radiating from any colour sample are
geodesic lines with uniform colour scales. Thus, OSA-UCS was adopted to develop
a CIE94-type colour-difference formula, valid under D65 illuminant and CIE
1964 colorimetric observer [40]. This formula was latter refined with chroma and
lightness compressions, achieving an Euclidean colour-difference formula based
also on the OSA-UCS space [41]. The equations conducting to this Euclidean
formula, noted as ΔEE , are as follows:

ΔEE = (ΔLE )2 + (ΔGE )2 + (ΔJE )2 (3.35)
   
LE = b1L ln 1 + baLL (10 LOSA ) with aL = 2.890, bL = 0.015 (3.36)

where OSA-UCS lightness, LOSA , which takes account of the Helmholtz–

Kohlrausch effect, is computed from the CIE 1964 chromaticity coordinates
x10 , y10 ,Y10 using the equations:
%    &
1/3 2 1
LOSA = 5.9 Y0 − + 0.042(Y0 − 30) 1/3
− 14.4 √ (3.37)
3 2

Y0 = Y10 4.4934 x10 + 4.3034 y10 − 4.2760 x10y10 − 1.3744x10
2 2

−2.5643y10 + 1.8103) (3.38)

and the coordinates GE and JE are defined from:

GE = −CE cos(h) (3.39)

JE = CE sin(h) (3.40)
   
CE = 1
bC ln 1 + baCC (10COSA ) with aC = 1.256, bC = 0.050 (3.41)
√
COSA = G2 + J 2 (3.42)
 
h = arctan − GJ (3.43)

with J and G coordinates defined, for the D65 illuminant, from the transformations:
   
J 2 (0.5735 LOSA + 7.0892) 0
=
G 0 −2 (0.7640 LOSA + 9.2521)
70 M. Melgosa et al.

⎛  ⎞
  ln A/B
0.1792 0.9837 ⎜ 0.9366 ⎟
⎝  ⎠ (3.44)
0.9482 −0.3175 ln B/C
0.9807
⎛ ⎞ ⎡ ⎤⎛ ⎞
A 0.6597 0.4492 −0.1089 X10
⎝ B ⎠ = ⎣ −0.3053 1.2126 0.0927 ⎦ ⎝ Y10 ⎠ (3.45)
C −0.0374 0.4795 0.5579 Z10

In 2008, Berns [13] proposed a series of colour-difference spaces based on

multi-stage colour vision theory and line integration. These colour spaces have
a similar transformation from tristimulus values to IPT space [42] to model
multi-stage colour vision theory. First, a CIECAM02’s chromatic adaptation trans-
formation ensures the colour appearance property in the first step of the model:
⎛ ⎞ ⎛ ⎞
XIlluminantE X
⎝ YIlluminant E ⎠ = M −1 MV K MCAT 02 ⎝ Y ⎠ (3.46)
CAT 02
ZIlluminant E Z

where tristimulus values ranged between 0 and 1 following the transformation.

MCAT 02 is the matrix employed in CIECAM02 [14] to transform XYZ to pseudo-
cone fundamentals, RGB. MV K is the von Kries diagonal matrix in RGB. Illuminant
E was selected because for either CIE standard observer, X = Y = Z = 1. Second,
a constrained linear transformation from tristimulus values to pseudo-cone funda-
mentals is performed to simulate the linear processing at the cones of the human
visual system:
⎛ ⎞ ⎛ ⎞ ⎛ ⎞
L e1 e2 e3 XIlluminantE
⎝ M ⎠ = ⎝ e4 e5 e6 ⎠ ⎝ YIlluminant E ⎠ (3.47)
S e7 e8 e9 cones ZIlluminant E
where (e1 + e2 + e3) = (e4 + e5 + e6 ) = (e7 + e8 + e9 ) = 1. These row sums were
optimization constraints and were required to maintain illuminant E as the reference
illuminant. Third, an exponential function was used for the nonlinear stage, where
γ defined the exponent (the same for all three cone fundamentals):
⎛ ⎞ ⎛ 1/γ ⎞
L L
⎝ M  ⎠ = ⎝ M 1/γ ⎠ (3.48)
S S1/γ

Fourth, the compressed cone responses were transformed to opponent signals:

⎛ ⎞ ⎛ ⎞⎛ ⎞ ⎛ ⎞
W  ⇔ K 100 0 0 o1 o2 o3 L
⎝ R ⇔ G ⎠ = ⎝ 0 100 0 ⎠ ⎝ o4 o5 o6 ⎠ ⎝ M ⎠ (3.49)
Y  ⇔ B 0 0 100 o7 o8 o9 opponency S
3 Colour Difference Evaluation 71

where (o1 + o2 + o3) = 1; (o4 + o5 + o6) = (o7 + o8 + o9 ) = 0. These rows constraints

generated an opponent-type system and were also used as optimization constraints.
The fifth step was to compress the chromaticness dimensions to compensate for the
chroma dependency (3.52):
⎛ ⎞ ⎛ ⎞
LE W  ⇔ K
⎝ aE ⎠ = ⎝ (R ⇔ G ) f (C) ⎠ (3.50)
bE (Y  ⇔ B ) f (C)

C = (R ⇔ G )2 + (Y  ⇔ B )2 (3.51)
ln(1+βCC )
f (C) = βCC
(3.52)

where C indicates the arithmetical average of the chroma of the two samples in the
colour pair.
Finally, Berns’ models adopted the Euclidean distance as the measure of colour
differences, and all the previous parameters were optimized [13] to achieve a
minimum deviation between visual and computed colour differences for the RIT-
DuPont dataset [6]:

ΔE E = (ΔLE )2 + (ΔaE )2 + (ΔbE )2 (3.53)

Recently, Shen and Berns [43] have developed an Euclidean colour space IPT-
EUC, claiming it as a potential candidate for a unique colour model for both
describing colour and measuring colour differences.
Euclidean colour spaces can be also developed by either analytical or compu-
tational methods to map the non-linear, non-uniform colour spaces to linear and
uniform colour spaces based on the clues provided by different colour difference
formulas optimized for reliable experimental datasets [44, 45].

3.4 Relationship Between Visual and Computed Colour

Differences

As stated at the beginning of this chapter, the main goal of a colour-difference

formula is to achieve a good relationship between what we see (ΔV ) and what
we measure (ΔE), for any two samples in colour space, viewed under any visual
conditions. Therefore, how to measure this relationship between subjective (ΔV )
and objective (ΔE) data is an important matter in this field. Although just a simple
plot of ΔEi against ΔVi (where i =1,. . . ,N indicates the number of colour pairs), may
be an useful tool, different mathematical indexes have been employed to measure
the strength of the relationship between ΔV and ΔE, as described in this section.
72 M. Melgosa et al.

PF/3 is a combined index which was proposed by Guan and Luo [7] from
previous metrics suggested by Luo and Rigg [26], which in turn employed the γ and
CV metrics proposed by Alder et al. [46] and the VAB metric proposed by Shultze
[47]. The corresponding defining equations are as follows:

 '   (2

1 N
ΔEi ΔEi
log10 (γ ) = ∑ log10
N i=1
− log10
ΔVi ΔVi
(3.54)


 N
 
2  ∑ ΔEi /ΔVi
1 N
(ΔEi − FΔVi )  i=1
VAB = ∑
N i=1 ΔEi FΔVi
with F = 
N
(3.55)
∑ ΔVi /ΔEi
i=1

 N
 ∑ ΔEi ΔVi
1 N
(ΔEi − f ΔVi ) 2
∑
i=1
CV = 100  2 with f = N
(3.56)
N i=1 ΔEi ∑ ΔVi2
i=1
 
100 (γ − 1) + VAB + CV
PF/3 = 100
(3.57)
3
where N indicates the number of colour pairs (with visual and computed differences
ΔVi and ΔEi , respectively), F and f are factors adjusting the ΔEi and ΔVi values
to the same scale, and the upper bar in a variable indicates the arithmetical mean.
For perfect agreement between ΔEi and ΔVi , CV and VAB should equal zero and γ
should equal one, in such a way that PF/3 should equal zero. A higher PF/3 value
indicates worse agreement. Guan and Luo [7] state that PF/3 gives roughly the
typical error in the predictions of ΔVi as a percentage: for example, a 30% error in
all pairs corresponds approximately to γ of 1.3, VAB of 0.3, and CV of 30 leading to
PF/3 of 30.
The decimal logarithm of γ is the standard deviation of the log10 (ΔEi /ΔVi ). This
metric was adopted because ΔEi values should be directly proportional to ΔVi , and
the ratio ΔEi /ΔVi should be constant. The standard deviation of the values of this
ratio could be used as a measure of agreement, but this would give rise to anomalies
which can be avoided by considering the logarithms of the ΔEi /ΔVi values [25].
Natural logarithms have sometimes been employed to define the γ index, but the
standard version of PF/3 uses decimal logarithms. The VAB and CV values express
the mean square root of the ΔEi values with respect to the ΔVi values (scaled by the
F or f coefficients), normalized to appropriate quantities. Therefore, the VAB and
CV indices could be interpreted as two coefficients of variations, and the F and f
factors as slopes of the plot of ΔEi against ΔVi (although they are not exactly the
slope of the linear-regression fit). In earlier papers [25, 26], the product-moment
correlation coefficient r was also employed as another useful measure to test the
relationship between ΔEi and ΔVi . However, the r coefficient was not included in
3 Colour Difference Evaluation 73

final PF/3 definition because it was found to be quite inconsistent with the other
three indices for different experimental and theoretical datasets [7,16,47]. The main
reason to propose the PF/3 index was that sometimes different measures led to
different conclusions; for example, one formula performed the best according to
CV while using VAB other different formula provided the most accurate prediction.
Thus, it was considered useful to avoid making a decision as to which of the metrics
was the best, and provide a single value to evaluate the strength of the relationship
between ΔEi and ΔVi [16]. Anyway, although PF/3 was widely employed in recent
colour-difference literature, other indices have been also employed in this field.
For example, the “wrong decision” percentage [48] is employed in acceptability
experiments, the coefficient of variation of tolerances was employed by Alman
et al. [49] and the linear correlation coefficients also continue being used by some
researchers [50, 51].
Any flaw in γ , CV, or VAB is immediately transferred to PF/3, which is by
definition an eclectic index. In addition, PF/3 cannot be used to indicate the
significance of the difference between two colour-difference formulas with respect
to a given set of visual data, because the statistical distribution followed by PF/3 is
unknown. This last point is an important shortcoming for the PF/3 index, because
the key question is not just to know that a colour-difference formula has a lower
PF/3 than other for a given set of reliable visual data, but to know whether these
two colour-difference formulas are or not statistically significant different for these
visual data. From the scientific point of view, it is not reasonable to propose a new
colour-difference formula if it is not significantly better than previous formulas, for
different reliable visual datasets. In addition, industry is reluctant to change colour-
difference formulas they are familiar to, in such a way that these changes must be
based on the achievement of statistically significant improvements.
In a recent paper [52] the STRESS index has been suggested as a good alternative
to PF/3 for colour-difference evaluation. STRESS comes from multidimensional
scaling [53], and is defined as follows:
) *
2 1/2
∑ (ΔEi − F1ΔVi ) ∑ ΔEi2
STRESS = 100 with F1 = (3.58)
∑ F1 ΔVi
2 2
∑ ΔEi ΔVi

It can be proved that STRESS2 is equal to (1 − r2 ), where r is equal to the

correlation coefficient when imposing the restriction that regression line should
pass through the origin (i.e., restricted regression). STRESS is always in the range
[0, 100], low STRESS values indicating good performance of a colour-difference
formula. But the key advantage of STRESS with respect to PF/3 is that F-test can
be performed to know on the statistical significance of the differences between two
colour-difference formulas A and B for a given set of visual data. Thus, the next
conclusions can be achieved from values of next parameter F:
 2
STRESSA
F= (3.59)
STRESSB
74 M. Melgosa et al.

OSA-GP

CAM02-UCS

CAM02-SCD

DIN99d

CIEDE2000

CIE94

CMC

CIELAB

0 10 20 30 40
STRESS

Fig. 3.4 Computed STRESS values using different colour-difference formulas for the combined
dataset employed at CIEDE2000 development [30]

• The colour-difference formula A is significantly better than B when F < FC

• The colour-difference formula A is significantly poorer than B when F > 1/FC
• The colour-difference formula A is insignificantly better than B when FC ≤ F < 1
• The colour-difference formula A is insignificantly poorer than B when 1 < F ≤
1/FC
• The colour-difference formula A is equal to B when F = 1
where FC is the critical value of the two-tailed F distribution with 95% confidence
level and (N − 1, N − 1) degrees of freedom.
STRESS can be also employed to measure inter- and intra-observer variability
[54]. STRESS values from reliable experimental datasets using different advanced
colour-difference formulas have been reported in the literature [55]. Thus, Fig. 3.4
shows STRESS values found for the combined dataset employed at CIEDE2000
development [30] (11,273 colour pairs), using the following colour-difference
formulas: CIELAB, [21] CMC, [25] CIE94, [27] CIEDE2000, [29] DIN99d, [35]
CAM02-SCD, [37] CAM02-UCS, [37] and OSA-GP [41]. For this combined
dataset, the worst colour-difference formula (highest STRESS) was CIELAB, and
the best (lowest STRESS) CIEDE2000. It can be added that, from F-test results,
for this specific dataset CIELAB performed significantly poorer than any of the
remaining colour-difference formulas, while CIEDE2000 was significantly better
than any of the remaining colour-difference formulas. Of course, different results
can be found for other experimental datasets [55], but best advanced colour-
difference formulas hardly produce STRESS values lower than 20, which should
be in part attributable to internal inconsistencies in the experimental datasets
employed. Methods for mathematical estimation of such kind of inconsistencies
3 Colour Difference Evaluation 75

have been suggested [56] concluding that, for the experimental dataset employed at
CIEDE2000 development [29, 30], only a few colour pairs with very small colour
differences have a low degree of consistency.

3.5 Colour Differences in Complex Images

Most complex images are not made up of large uniform fields. Therefore,
discrimination and appearance of fine patterned colour images differ from similar
measurements made using large homogeneous fields [57]. Direct application
of previously mentioned colour-difference formulas to predict complex image
difference (e.g., using a simple pixel by pixel comparison method) does not give
satisfactory results. Colour discrimination and appearance is a function of spatial
pattern. In general, as the spatial frequency of the target goes up (finer variations
in space), colour differences become harder to see, especially differences along the
blue-yellow direction. So, if we want to apply a colour-difference formula to colour
images, the patterns of the image have to be taken into account.
Different spatial colour-difference metrics have been suggested, the most famous
one being the one proposed by Zhang and Wandell [58] in 1996, known as S-
CIELAB. S-CIELAB is a “perceptual colour fidelity” metric. It measures how
accurate the reproduction of a colour image is to the original when viewed by a
human observer. S-CIELAB is a spatial extension of CIELAB, where two input
images are processed like made in the human visual system, before conventional
CIELAB colour differences are applied pixel by pixel. Specifically, the steps
followed by S-CIELAB are as follows: (1) each pixel (X,Y, Z) in the input images
is translated to an opponent colour space, consisting of one luminance and two
chrominance components; (2) each one of these three components is passed through
a spatial filter that is selected according to the spatial sensitivity of the human visual
system to this component, taken into account visual conditions; (3) the filtered
images are transformed back into the CIE X,Y, Z format; (4) finally, the colour
differences can be computed using the conventional CIELAB colour-difference
formula, and the average of these colour differences for all pixels could then be
used to represent the difference between two complex images. In fact, this idea can
be applied using at the end any colour-difference formula; for example, in 2003
Johnson and Fairchild [59] applied the S-CIELAB framework replacing CIELAB
by the CIEDE2000 colour-difference formula. Recently, Johnson et al. [60] have
also pointed out that, for image difference calculations, the ideal opponent colour
space would be both linear and orthogonal, such that the linear filtering is correct
and any spatial processing on one channel does not affect the others, proposing a
new opponent colour space and corresponding spatial filters specifically designed
for image colour-difference calculations.
The evaluation of colour differences in complex images requires the corre-
sponding images be carefully selected, as suggested by standardization organisms,
avoiding potential bias from some kind of images [61]. Experimental methods
76 M. Melgosa et al.

employed to compare image quality must also be carefully considered [62]. While
some results indicate a clear advantage of S-CIELAB with respect to CIELAB
analyzing colour differences in complex images [63], other results [64] suggest no
clear improvements using spatial colour-difference models, and results dependent
on image content. Recent CIE Publication 199–2011 [65] provides useful informa-
tion related to methods for evaluating colour differences in images.

3.6 Future Directions

Colour differences have been an active field of research since the 1950s trying
to respond to industrial requirements in important topics like colour control,
colour reproduction, etc. CIE-proposed colour-difference formulas have played an
important positive role in the communication between buyers and sellers, as well as
among different industries. The CIE recommendations of CIE94 and CIEDE2000
colour-difference formulas in 1995 and 2001, respectively, are eloquent examples
of significant work and advances in this scientific area. Currently, research on
colour differences continues, in particular, within some CIE Technical Committees
in Divisions 1 and 8, as shown by the following examples: CIE TC1–55 (chairman:
M. Melgosa) is working on the potential proposal of a uniform colour space
for industrial colour-difference evaluation; CIE TC1–57 (chairman: A. Robertson)
“Standards in colorimetry” has proposed the CIEDE2000 colour-difference formula
as a CIE standard; CIE TC1–63 (chairman: K. Richter) has studied the range of
validity of the CIEDE2000 colour-difference formula, concluding with the proposal
of the new CIE TC1–81 (chairman: K. Richter) to analyze the performance of
colour-difference formulas for very small colour differences (visual thresholds); CIE
TC8–02 (chairman: M.R. Luo) studied colour differences in complex images [65].
Another important aspect in colour-difference research is the need of new
reliable experimental datasets which can be used to develop better colour-difference
formulas. New careful determinations of visual colour differences under well-
defined visual conditions, together with their corresponding uncertainties, are highly
desirable [66]. At the same time it is also very convenient to avoid an indiscriminate
use of new colour-difference formulas, which should affect negatively industrial
colour communication. New colour-difference formulas are only interesting if they
can prove a statistically significant improvement with respect to previous ones, for
several reliable experimental datasets.
There is an increasing activity aimed at incorporating colour-appearance models
into practical colour-difference specification. For example, a colour appearance
model could incorporate the effects of the background and luminance level on
colour-difference perception, in such a way that the associated colour-difference
formula could be applied to a wide set of visual conditions, in place of just a
given set of “reference conditions”. A colour appearance model would also make
it possible to directly compare colour differences measured for different viewing
conditions or different observers. Colour appearance models would also make it
3 Colour Difference Evaluation 77

possible to calculate colour differences between a sample viewed in one condition

and a second sample viewed in another different condition. As stated by Fairchild
[15], “it is reasonable to expect that a colour difference equation could be optimized
in a colour appearance space, like CIECAM02, with performance equal to, or better
than equations like CIE94 and CIEDE2000.”
In many situations colour is the most important attribute of objects’ visual
appearance, but certainly it is not the only one. At least, gloss, translucency, and
texture may interact with colour and contribute to the so-called “total difference”.
Total difference models including colour differences plus coarseness or glint
differences have been proposed in recent literature [67, 68].

Acknowledgments To our CIMET Erasmus-Mundus Master students ([Link]

[Link]/) enrolled in the “Advanced Colorimetry” course during the academic
years 2008–2009 and 2009–2010, who contributed with their questions and comments to improve
our knowledge in the field of colour-difference evaluation. This work was partly supported by
research project FIS2010–19839, Ministerio de Educación y Ciencia (Spain), with European
Regional Development Fund (ERDF).

References

1. CIE Publication 101 (1993) Parametric effects in colour-difference evaluation. CIE Central
Bureau, Vienna
2. Witt K (1995) CIE guidelines for coordinated future work on industrial colour-difference
evaluation. Color Res Appl 20:399–403
3. Kuehni RG (2009) Variability in estimation of suprathreshold small color differences. Color
Res Appl 34:367–374
4. MacAdam DL (1942) Visual sensitivities to color differences in daylight. J Opt Soc Am
32:247–274
5. Shen S, Berns RS (2011) Color-difference formula performance for several datasets of small
color differences based on visual uncertainty. Color Res Appl 36:15–26
6. Berns RS, Alman DH, Reniff L, Snyder GD, Balonon-Rosen MR (1991) Visual determi-
nation of suprathreshold color-difference tolerances using probit analysis. Color Res Appl
16:297–316
7. Guan S, Luo MR (1999) Investigation of parametric effects using small colour-differences.
Color Res Appl 24:331–343
8. ISO 105-A02:1993 Tests for Colour Fastness-Part A02: Gray Scale for Assessing Change
in Colour, International Organization for Standardization Geneva, Switzerland. [Link]
[Link]
9. AATCC Committee RA36, AATCC Evaluation Procedure 1 (2007) Gray scale for color
change. AATCC, NC, Research Triangle Park. [Link]
10. Fastness Tests Co-ordinating Committee (F.T.C.C.) Publication XI (1953) The development of
the geometric grey scales for fastness assessment. J Soc Dyers Colour 69:404–409
11. Cárdenas LM, Shamey R, Hinks D (2009) Development of a novel linear gray scale for visual
assessment of small color differences. AATCC Review 9:42–47
12. Montag ED, Wilber DC (2003) A comparison of color stimuli and gray-scale methods of color
difference scaling. Color Res Appl 28:36–44
13. Berns RS (2008) Generalized industrial color-difference based on multi-stage color vision and
line-element integration. Óptica Pur Appl 41:301–311
78 M. Melgosa et al.

14. CIE Publication 159:2004 (2004) A colour appearance model for colour management systems:
CIECAM02. CIE Central Bureau, Vienna
15. Fairchild MD (2005) Colour Appearance Models, 2nd edn. Wiley, New York
16. Luo MR (2002) Development of colour-difference formulae. Rev Prog Color 32:28–39
17. McDonald R (1982) A review of the relationship between visual and instrumental assessment
of colour difference, part 1. J Oil Colour Chem Assoc 65:43–53
18. McDonald R (1982) A review of the relationship between visual and instrumental assessment
of colour difference, part 2. J Oil Colour Chem Assoc 65:93–106
19. Wit K (2007) CIE color difference metrics. In: Schanda J (ed) Chapter 4 in Colorimetry-
Understanding the CIE System, Wiley, New York
20. CIE Publication 13.3 (1995) Method of measuring and specifying colour rendering properties
of light sources. CIE Central Bureau, Vienna
21. CIE 15:2004 (2004) Colorimetry, 3rd edn. CIE Central Bureau, Vienna
22. Robertson AR (1990) Historical development of CIE recommended color difference equations.
Color Res Appl 15:167–170
23. Kuehni RG (1990) Industrial color-difference: progress and problems. Color Res
Appl 15:261–265
24. Melgosa M (2000) Testing CIELAB-based color-difference formulas. Color Res
Appl 25:49–55
25. Clarke FJJ, McDonald R, Rigg B (1984) Modification to the JPC79 colour-difference formula.
J Soc Dyers Colour 100:128–132
26. Luo MR, Rigg B (1987) BFD(l:c) colour-difference formula. Part 1 – Development of the
formula. J Soc Dyers Colour 103:86–94
27. CIE Publication 116 (1995) Industrial colour-difference evaluation. CIE Central Bureau,
Vienna
28. McDonald R (1974) The effect of non-uniformity in the ANLAB color space on the
interpretation of visual colour differences. J Soc Dyers Colour 90:189–198
29. CIE Publication 142 (2001) Improvement to industrial colour-difference evaluation. CIE
Central Bureau, Vienna
30. Luo MR, Cui G, Rigg B (2001) The development of the CIE 2000 colour-difference formula:
CIEDE2000. Color Res Appl 26:340–350
31. Melgosa M, Huertas R, Berns RS (2004) Relative significance of the terms in the CIEDE2000
and CIE94 color-difference formulas. J Opt Soc Am A 21:2269–2275
32. Sharma G, Wu W, Dalal EN (2005) The CIEDE2000 color-difference formula: implementation
notes, supplementary test data, and mathematical observations. Color Res Appl 30:21–30
33. Kuehni RG (2002) CIEDE2000, milestone or final answer? Color Res Appl 27:126–128
34. 6176 DIN (2000) Farbmetrische Bestimmung von Farbabständen bei Köroerfarben nach der
DIN-99-Formel. DIN Deutsche Institut für Normung e.V, Berlin
35. Cui G, Luo MR, Rigg B, Roesler G, Witt K (2002) Uniform colour spaces based on the DIN99
colour-difference formula. Color Res Appl 27:282–290
36. Kuehni RG (1999) Towards an improved uniform color space. Color Res Appl 24:253–265
37. Luo MR, Cui G, Li C (2006) Uniform colour spaces based on CIECAM02 colour appearance
model. Color Res Appl 31:320–330
38. Xue Y (2008) Uniform color spaces based on CIECAM02 and IPT color difference equations.
MD Thesis, Rochester Institute of Technology, Rochester, NY
39. MacAdam DL (1974) Uniform color scales. J Opt Soc Am 64:1691–1702
40. Huertas R, Melgosa M, Oleari C (2006) Performance of a color-difference formula based on
OSA-UCS space using small-medium color differences. J Opt Soc Am A 23:2077–2084
41. Oleari C, Melgosa M, Huertas R (2009) Euclidean color-difference formula for small-medium
color differences in log-compressed OSA-UCS space. J Opt Soc Am A 26:121–134
42. Ebner F, Fairchild MD (1998) Development and testing of a color space (IPT) with improved
hue uniformity. In: Proceedings of 6th Color Imaging Conference, 8–13, IS&T, Scottsdale, AZ
43. Shen S (2008) Color difference formula and uniform color space modeling and evaluation. MD
Thesis, Rochester Institute of Technology, Rochester, NY
3 Colour Difference Evaluation 79

44. Thomsen K (2000) A Euclidean color space in high agreement with the CIE94 color difference
formula. Color Res Appl 25:64–65
45. Urban P, Rosen MR, Berns RS, Schleicher D (2007) Embedding non-euclidean color
spaces into Euclidean color spaces with minimal isometric disagreement. J Opt Soc Am A
24:1516–1528
46. Alder C, Chaing KP, Chong TF, Coates E, Khalili AA, Rigg B (1982) Uniform chromaticity
scales – New experimental data. J Soc Dyers Colour 98:14–20
47. Schultze W The usefulness of colour-difference formulae for fixing colour tolerances. In:
Proceedings of AIC/Holland (Soesterberg 1972) 254–265
48. McLaren K (1970) Colour passing—Visual or instrumental? J Soc Dyers Colour 86:389–392
49. Alman DH, Berns RS, Snyder GD, Larsen WA (1989) Performance testing of color-difference
metrics using a color tolerance dataset. Color Res Appl 14:139–151
50. Gibert JM, Dagà JM, Gilabert EJ, Valldeperas J and the Colorimetry Group (2005) Evaluation
of colour difference formulae. Color Technol 121:147–152
51. Attridge GG, Pointer MR (2000) Some aspects of the visual scaling of large colour differences-
II. Color Res. Appl 25:116–122
52. Garcı́a PA, Huertas R, Melgosa M, Cui G (2007) Measurement of the relationship between
perceived and computed color differences. J Opt Soc Am A 24:1823–1829
53. Coxon APM (1982) The user’s guide to multidimensional scaling. London, Heinemann
54. Melgosa M, Garcı́a PA, Gómez-Robledo L, Shamey R, Hinks D, Cui G, Luo MR (2011) Notes
on the application of the standardized residual sum of squares index for the assessment of intra-
and inter-observer variability in color-difference experiments. J. Opt. Soc. Am. A. 28:949–953
55. Melgosa M, Huertas R, Berns RS (2008) Performance of recent advanced color-difference
formulas using the standardized residual sum of squares index. J. Opt. Soc. Am. A 25:1828–
1834
56. Morillas S, Gómez-Robledo L, Huertas R, Melgosa M (2009) Fuzzy analysis for detection of
inconsistent data in experimental datasets employed at the development of the CIEDE2000
colour difference formula. J Mod Optic 56:1447–1456
57. Wandell BA (1996) Photoreceptor sensitivity changes explain color appearance shifts induced
by large uniform background in dichoptic matching. Vis Res 35:239–254
58. Zhang XM, Wandell BA (1996) A spatial extension to CIELAB for digital color image
reproduction. Proc Soc Information Display 27:731–734
59. Johnson GM, Fairchild MD (2003) A top down description of S-CIELAB and CIEDE2000.
Color Res Appl 28:425–435
60. Johnson GM, Song X, Montag E, Fairchild MD (2010) Derivation of a color space for image
color difference measurements. Color Res Appl 35:387–400
61. International Standardization Organization (ISO) Graphic technology—Prepress digital data
exchange. Part 1, ISO 12640–1 (1997), Part 2, ISO 12640–2 (2004), Part 3 ISO 12640–3 (2007)
62. International Standardization Organization (ISO) (2005) Photography—Psychophysical exper-
imental method to estimate image quality. Parts 1, 2 and 3, ISO 20462
63. Aldaba MA, Linhares JM, Pinto PD, Nascimento SM, Amano K, Foster DH (2006) Visual
sensitivity to color errors in images of natural scenes. Vis Neurosci 23:555–559
64. Lee DG (2008) A colour-difference model for complex images on displays. Ph.D. Thesis,
University of Leeds, UK
65. CIE Publication 199:2011 (2011) Methods for evaluating colour differences in images. CIE
Central Bureau, Vienna
66. Melgosa M (2007) Request for existing experimental datasets on color differences. Color Res
Appl 32:159
67. Huang Z, Xu H, Luo MR, Cui G, Feng H (2010) Assessing total differences for effective
samples having variations in color, coarseness, and glint. Chinese Optics Letters 8:717–720
68. Dekker N, Kirchner EJJ, Supèr R, van den Kieboom GJ, Gottenbos R (2011) Total appearance
differences for metallic and pearlescent materials: Contributions from color and texture. Color
Res Appl 36:4–14
Chapter 4
Cross-Media Color Reproduction and Display
Characterization

Jean-Baptiste Thomas, Jon Y. Hardeberg, and Alain Trémeau

The purest and most thoughtful minds are those which love
color the most
John Ruskin

Abstract In this chapter, we present the problem of cross-media color

reproduction, that is, how to achieve consistent reproduction of images in different
media with different technologies. Of particular relevance for the color image
processing community is displays, whose color properties have not been extensively
covered in previous literature. Therefore, we go more in depth concerning how to
model displays in order to achieve colorimetric consistency.
The structure of this chapter is as follows: After a short introduction, we
introduce the field of cross-media color reproduction, including a brief description
of current standards for color management, the concept of colorimetric characteri-
zation of imaging devices, and color gamut mapping. Then, we focus on state of the
art and recent research in the colorimetric characterization of displays. We continue
by considering methods for inverting display characterization models; this is an
essential step in cross-media color reproduction, before discussing briefly quality
factors, based on colorimetric indicators. Finally, we draw some conclusions and
outline some directions for further research.

J.-B. Thomas ()

Laboratoire Electronique, Informatique et Image, Université de Bourgogne, Dijon, France
e-mail: [Link]@[Link]
J.Y. Hardeberg
The Norwegian Color Research Laboratory, Gjøvik University College, Gjøvik, Norway
e-mail: [Link]@[Link]
A. Trémeau
Laboratory Hubert Curien, UMR CNRS 5516, Jean Monnet University, Saint-Etienne, France
e-mail: [Link]@[Link]

C. Fernandez-Maloigne (ed.), Advanced Color Image Processing and Analysis, 81

DOI 10.1007/978-1-4419-6190-7 4,
© Springer Science+Business Media New York 2013
82 J.-B. Thomas et al.

Keywords Color management • Cross-media color reproduction • Colorimetric

device characterization • Gamut mapping • Displays • Inverse model

4.1 Introduction

Digital images today are captured and reproduced using a plethora of different
imaging technologies (e.g., digital still cameras based on CMOS or CCD sensors,
Plasma or Liquid Crystal Displays, inkjet, or laser printers). Even within the
same type of imaging technology, there are many parameters which influence the
processes, resulting in a large variation in the color behavior of these devices.
It is therefore a challenge to achieve color consistency throughout an image
reproduction workflow, even more so since such image reproduction workflows tend
to be highly distributed and generally uncontrolled. This challenge is relevant for a
wide range of users, from amateurs of photography to professionals of the printing
industry. And as we try to advocate in this chapter, it is also highly relevant to
researchers within the field of image processing and analysis.
In the next section we introduce the field of cross-media color reproduction,
including a brief description of current standards for color management, the
concept of colorimetric characterization of imaging devices, and color gamut
mapping. Then, in Sect. 4.3 we focus on state of the art and recent research in
the characterization of displays. In Sect. 4.4, we consider methods for inverting
display characterization models; this is an essential step in cross-media color
reproduction, before discussing quality factors, based on colorimetric indicators,
briefly in Sect. 4.5. Finally, in Sect. 12.5 we draw some conclusions and outline
some directions for further research.

4.2 Cross-Media Color Reproduction

When using computers and digital media technology to acquire, store, process,
and reproduce images of colored objects or scenes, a digital color space is used,
typically RGB, describing each color as a combination of variable amounts of the
primaries red, green, and blue. Since most imaging devices speak RGB one may
think that there is no problem with this. However, every individual device has its
own definition of RGB, i.e., for instance for output devices such as displays, for
the same input RGB values, different devices will produce significantly different
colors. It usually suffices to enter the TV section of an home electronics store to be
reminded of this fact.
So, therefore, the RGB color space is usually not standardized, and every individ-
ual imaging device has its own definition of it, i.e., its very own relationship between
the displayed or acquired real-world color and the corresponding RGB digital color
space. Achieving color consistency throughout a complex and distributed color
4 Cross-Media Color Reproduction and Display Characterization 83

reproduction workflow with several input and output devices is therefore a serious
challenge; achieving such consistency defines the research field of cross-media color
reproduction.
The main problem is thus to determine the relationships between the different
devices’s color languages, analogously to color dictionaries. As we will see in the
next sections, a standard framework has been defined (color management system),
in which dictionaries (profiles) are defined for all devices; between their native color
language and a common, device-independent language. Defining these dictionaries
by characterizing the device’s behavior is described in Sect. 4.2.2, while Sect. 4.2.3
addresses the problem of when a device simply does not have rich enough
vocabulary to reproduce the colors of a certain image.

4.2.1 Color Management Systems

By calibrating color peripherals to a common standard, color management system

(CMS) software and architecture makes it easier to match colors that are scanned to
those that appear on the monitor and printer, and also to match colors designed on
the monitor, using, e.g., CAD software, to a printed document. Color management
is highly relevant to persons using computers for working with art, architecture,
desktop publishing or photography, but also to non-professionals, as, e.g., when
displaying and printing images downloaded from the Internet or from a Photo CD.
To obtain faithful color reproduction, a CMS has two main tasks. First, colori-
metric characterization of the peripherals is needed, so that the device-dependent
color representations of the scanner, the printer, and the monitor can be linked to
a device-independent color space, the profile connection space (PCS). This is the
process of profiling. Furthermore, efficient means for processing and converting
images between different representations are needed. This task is undertaken by
the color management module (CMM).
The industry adoption of new technologies such as CMS depends strongly on
standardization. The international color consortium (ICC, [Link]
plays a very important role in this concern. The ICC was established in 1993 by
eight industry vendors for the purpose of creating, promoting, and encouraging
the standardization and evolution of an open, vendor-neutral, cross-platform CMS
architecture and components.
For further information about color management system architecture, as well as
theory and practice of successful color management, refer to the ICC specification
[47] or any recent textbooks on the subject [40].
Today there is wide acceptance of the ICC standards, and different studies
such as one by [71] have concluded that color management solutions offered by
different vendors are approximately equal, and that color management has passed
the breakthrough phase and can be considered a valid and useful tool in color image
reproduction.
84 J.-B. Thomas et al.

However, there is still a long way to go, when it comes to software development
(integration of CMS in operating systems, user-friendliness, simplicity, etc.), re-
search in cross-media color reproduction (better color consistency, gamut mapping,
color appearance models, etc.), and standardization. Color management is a very
active area of research and development, though limited by our knowledge on
the human perception process. Thus in the next sections, we will briefly review
different approaches to the colorimetric characterization of image acquisition and
reproduction devices.

4.2.2 Device Colorimetric Characterization

Successful cross-media color reproduction needs the calibration and the character-
ization of each color device. It further needs a color conversion algorithm, which
permits to convert color values from one device to another.
In the literature, the distinction between calibration and characterization can
vary substantially, but the main idea usually remains the same. For instance, some
authors will consider a tone response curve establishment as a part of the calibration,
others as a part of the characterization. These difference does not mean too much in
practice and is just a matter of terminology. Let us consider the following definition:
The calibration process put a device in a fixed state, which will not change with
time. For a color device, it consists in setting up the device. Settings can be position,
brightness, contrast, and sometimes primaries and gamma, etc.
The characterization process can be defined as understanding and modeling the
relationship between the input and the output, in order to control a device for a given
calibration set-up. For a digital color device, this means either to understand the
relationship between a digital value input and a produced color for an output color
device (printer, display) or, in the case of an input color device (camera, scanner), to
understand the relationship between the acquired color and the digital output value.
Usually, a characterization model is mostly static, and is relying on the capability of
the device to remain in a fixed state, thus on the calibration step.
As stated above, the characterization of a color device is a modeling step, which
permits to relate the digital value that characterizes the device and the actual color
defined in a standard color space, such as CIEXYZ. There are different approaches
to modeling a device.
One can consider a physical approach, which will aim to determine a set of
physical parameters of a device, and uses these in a physical model based on
the technology definition. Such an approach has been extensively used for CRT
displays, and also it is quite common for cameras. In this case, the resulting
accuracy will be constrained by how well the device fits the model hypothesis and
how accurate the related measurements were taken. Commonly a physical device
model consists in a two steps process. First, a linearization of the intensity response
curves of the individual channels, i.e., the relation between the digital value and
4 Cross-Media Color Reproduction and Display Characterization 85

the corresponding intensity of light. The second step is typically colorimetric linear
transform (i.e., a 3x3 matrix multiplication). The characteristics of the colorimetric
transform is based on the chromaticity of the device primaries.
Another approach consists in fitting a data set with any numerical model. In this
case, the accuracy will depend on the number of data, on their distribution and on
the interpolation method used. Typically a numerical model would require more
measurement, but would make no assumption on the device behavior. We can note
that the success of such a model will depend also on the capacity of the model to fit
with the technology anyway.
For a numerical method, depending on the interpolation method used, one have
to provide different sets of measures in order to optimize the model determination.
This implies to first define which color space is used to make all the measures.
The CIEXYZ color space seems at first to be the best choice considering that some
numerical method would use its vectorial space properties successfully, particularly
additivity, in opposition with CIELAB. An advantage is that it is absolute and can
be used as an intermediary color space to a uniform color space, CIELAB, which is
recommended by the CIE for measuring the color difference when we will evaluate
the model accuracy (the Δ E in CIELAB color space). However, since we define the
error of the model, and often the cost function of the optimization process as an
Euclidean distance in CIELAB, this color space can be a better choice.
These sets of measures can be provided using a specific (optimal) color chart, or
a first approach can be to use a generic color chart, which allows to define a first
model of characterization.
However, it has been shown that it is of major importance to have a good
distribution of the data everywhere in the gamut of the device and more particularly
on the faces and the edges of the gamut, which is roughly fitting with the edges and
faces of the RGB-associated cube. These faces and edges define the color gamut
of the color device. The problem with acquisition device such as cameras is that
the lighting conditions are changing, and it is difficult to have a dedicated data set
of patches to measure for every possible condition. Thus, optimized color charts
have been designed, for which the spectral characteristics of the color patches are
designed carefully.
Another possibility is that, based on a first rough or draft model, one can provide
an optimal data set to measure, which takes into account the nonlinearity of the
input device. There are several methods to minimize errors due to the nonlinear
response of devices. By increasing the number of patches, we can tighten the mesh’s
sampling. This method can be used to reach a lower error. Unfortunately, it might
not improve much the maximum error. To reduce it, one can decide to over-sample
some particular area of the color space. The maximum error is on the boundaries
of the gamut, since there are fewer points to interpolate, and in the low luminosity
areas, as our eyes can easily see small color differences in dark colors. Finally, one
can solve this nonlinearity problem by using a nonlinear data set distribution, which
provides a quite regular sampling in the CIELAB color space.
86 J.-B. Thomas et al.

[Link] Characterization of Input Devices

An input device has the ability to transform color information of a scene or an

original object into digital values. A list of such devices would include digital still
cameras, scanners, camcorders, etc. The way it transforms the color information is
usually based on (three) spectral filters with their highest transmission or resulting
color around a Red, Green, and Blue part of the spectrum. The intensity for each
filter will be related to the RGB values. A common physical model of such a device
is given as

ρ = f (ν ) = L(λ )R(λ )S(λ )dλ , (4.1)

where ρ is the actual digital value output, ν is the nonlinearized value, L(λ ), R(λ ),
S(λ ) are the spectral power distribution of the illuminant, the spectral reflectance of
the object, and the spectral sensitivity of the sensor, including a color filter.
The input device calibration includes the setup of the time exposure, the
illumination (for a scanner), the contrast setup, the color filters, etc.
In the case of input devices, let us call the forward transform the transform which
relates the acquired color with the digital value, e.g., conversion from CIEXYZ to
RGB. Meanwhile the inverse transform will estimate the acquired color given digital
value caught by the device, e.g., converts from RGB to CIEXYZ.
The input device characterization can be done using a physical modeling or a
combination of numerical methods. In the case of a physical modeling, the tone
response curves will have to be retrieved; the spectral transmission of the color
filters may have to be retrieved too, in order to determine their chromaticities, thus
establishing the linear transform between intensity linearized values and the digital
values. This last part requires usually a lot of measurements, and may require the use
of a monochromator or an equivalent expensive tool. In order to reduce this set of
measurements, one needs to make some assumptions and to set some constraints to
solve the related inverse problem. Such constraints can be the modality of the spec-
tral response of a sensor or that the sensor response curve can be fitted with just a
few of the first Fourier coefficients, see, e.g., [10,36,76]. Such models would mostly
use the CIEXYZ color space or another space which has the additivity property.
Johnson [51] gives good advice for achieving a reliable color transformation for
both scanners and digital cameras. In his paper, one can find diverse characterization
procedures, based on the camera colorimetric evaluation using a set of test images.
The best is to find a linear relationship to map the output values to the input
target (each color patch). The characterization matrix, once more, provides the
transformation applied to the color in the image. In many cases, the regression
analysis shows that the first order linear relationship is not satisfactory and a
higher order relationship or even nonlinear processing is required (log data, gamma
correction, or S-shape, e.g.). Lastly, if a matrix cannot provide the transformation,
then a look-up table (LUT) will be used. Unfortunately, the forward transform can
be complicated and quite often produces artifacts [51]. Possible solutions to the
problems of linear transformations encountered by Johnson are least-squares fitting,
4 Cross-Media Color Reproduction and Display Characterization 87

nonlinear transformations, or look-up tables with interpolation. In the last case, any
scanned pixel can be converted into tristimulus values via the look-up table(s) and
interpolation is used for intermediate points which do not fall in the table itself. This
method is convenient for applying a color transformation when a first order solution
is not relevant. It can have a very high accuracy level if the colors are properly
selected.
The colorimetric characterization of a digital camera was analyzed by [45].
An investigation was done to determine the influence of the polynomial used
for interpolation and the possible correlation between the RGB channels. The
channel independence allows us to separate the contribution of spectral radiance
from the three channels. Hong et al. [45] also checked the precision of the model
with respect to the training samples’ data size provided and the importance of
the color precision being either 8 or 12 bits. According to the authors, there are
two categories of color characterization methods: either spectral sensitivity based
(linking the spectral sensitivity to the CIE color-matching functions) or color target
based (linking color patches to the CIE color-matching functions). These two
solutions lead to the same results, but the methods and devices used are different.
Spectral sensitivity analysis requires special equipment like a radiance meter and a
monochromator; while a spectrophotometer is the only device needed for the color
target-based solution. Typical methods like 3D look-up tables with interpolation
and extrapolation, least square polynomials modeling and neural networks can be
used for the transformation between RGB and CIEXYZ values, but in this article,
polynomial regression is used. As for each experiment only one parameter (like
polynomial order, number of quantization levels, or size of the training sample)
changes, the Δ Eab∗ difference is directly linked to the parameter.

Articles published on this topic are rare, but characterization of other input
devices with a digital output operates the same way. Noriega et al. [67] and
[37] further propose different transformation techniques. These articles discuss the
colorimetric characterization of a scanner and a negative film. In the first article
[67], the authors decided to use least squares fitting, LUTs and distance-weighted
interpolation. The originality comes from the use of the Mahalanobis distance used
to perform the interpolation. The second article [37] deals with the negative film
characterization. Distance-weighted interpolation, Gaussian interpolation neural
networks, and nonlinear models have been compared using Principal Component
Analysis. In these respective studies, the models were trained with the Mahalanobis
distance (still using the color difference as a cost function) and neural networks.

[Link] Characterization of Output Devices

An output device in this context is any device that will reproduce a color, such as
printers, projection systems, or monitors. In this case, the input to the device is a
digital value, and we will call the forward transform the transform that predicts the
color displayed for a given input, e.g., RGB to CIEXYZ. The inverse or backward
transform will then define which digital value we have to input to the device to
reproduce a wanted color, e.g., CIEXYZ to RGB.
88 J.-B. Thomas et al.

The characterization approach for output devices and media is similar to that of
input devices. One has to determine a model based on more or less knowledge of
the physical behavior of the device, and more or less measurement of color patches
and mathematical approximation/interpolation. Since displays are covered in depth
in Sect. 4.3, we will here briefly discuss printer characterization.
We can distinguish between two kinds of printer characterization models, the
computational and the physical ones. Typically, for a 4-colorant CMYK printer
the computational approach consists in building a grid in four dimensions, a
multidimensional look-up table (mLUT). The estimation of the resulting color
for a given colorant combination will then be calculated by multidimensional
interpolation in the mLUT. An important design trade-off for such modeling is
between the size of the mLUT and the accuracy of the interpolation.
The physical models attempt to imitate the physics involved in the printing
device. Here also these models can be classified into two subtypes with regard to
the assumptions they make and their complexity [90]: regression-based and first-
principal models. Regression-based models are rather simple and works with a few
parameters to predict a printer output while first-principal model will closely imitate
the physics of the printing process by taking into account multiple light interactions
between the paper and the ink layers, for instance. Regression-based models are
commonly used to model the behavior of digital printing devices.
During the last century, printing technology has evolved and the printer models
as well. Starting from a single-colorant printing device the Murray-Davies model
predicts the output spectral reflectances of a single-colorant coverage value knowing
the spectral reflectance of the paper and maximum colorant coverage value. This
model was extended to color by [64]. The prediction of a colorant combination is
the summation of all the colorants involved in the printing process weighted by
their coverage on the paper. All the colorants are referring to all the primaries (cyan,
magenta, and yellow in case of a CMY printer) plus all the combination between
them plus the paper, these colors are called the Neugebauer primaries (NP). Later
the interaction of light penetrating and scattering into the paper was added to these
models by [95], as form of an exponent known as the n factor. For more information
about printer characterization, refer, e.g., to [38].

4.2.3 Color Gamut Considerations

A color gamut is the set of all colors that can be produced by a given device or
that are present in a given image. Although these sets are in principle discrete,
gamuts are most often represented as volumes or blobs in a 3D color space using a
gamut boundary descriptor [7]. When images are to be reproduced between different
devices, the problem of gamut mismatch has to be addressed. This is usually referred
to as color gamut mapping. There is a vast amount of literature about the gamut-
mapping problem, see, for instance, a recent book by [63].
4 Cross-Media Color Reproduction and Display Characterization 89

To keep the image appearance, some constraints are usually considered while
doing a gamut mapping:
• Preserve the gray axis of the image and aim for maximum luminance contrast.
• Reduce the number of out-of-gamut colors.
• Minimize hue shifts.
• Increase the saturation.
CIELAB is one of the most often used color spaces for gamut mapping, but there
are deficiencies in the uniformity of hue angles in the blue region. To prevent this
shift, one can use Hung and Berns’ data to correct the CIELAB color space [21].
To map a larger source gamut into a smaller destination gamut of a device with
a reduced lightness dynamic range, often a linear lightness remapping process is
applied. It suffers from a global reduction in the perceived lightness contrast and an
increase in the average lightness of the remapped image. It is of utmost importance
to preserve the lightness contrast. An adaptive lightness rescaling process has been
developed by [22]. The lightness contrast of the original scene is increased before
the dynamic range compression is applied to fit the input lightness range into the
destination gamut. This process is known as a sigmoidal mapping function, the
shape of this function aids in the dynamic range mapping process by increasing
the image contrast and by reducing the low-end textural defects of hard clipping.
We can categorize different types of pointwise gamut-mapping technics (See
Fig. 4.1); gamut clipping only changes the colors outside the reproduction gamut
while gamut compression changes all colors from the original gamut. The knee
function rescaling preserves the chromatic signal through the central portion of the
gamut, while compressing the chromatic signal near the edges of the gamut. The
sigmoid-like chroma mapping function has three linear segments; the first segment
preserves the contrast and colorimetry, the second segment is a mid-chroma boost
(increasing chroma), and the last segment compresses the out-of-gamut chroma
values into the destination gamut.
Spatial gamut mapping has become an active field of research in the recent years
[35, 56]. In contrast to the conventional color gamut-mapping algorithms, where
the mapping can be performed once and for all and stored as a look-up table, e.g.,
in an ICC profile, the spatial algorithms are image dependent by nature. Thus,
the algorithms have to be applied for every single image to be reproduced, and
make direct use of the gamut boundary descriptors many times during the mapping
process.
Quality assessment is also required for the evaluation of gamut-mapping
algorithms, and extensive work has been carried out on subjective assessment
[32]. This evaluation is long, tiresome, and even expensive. Therefore, objective
assessment methods are preferable. Existing work on this involves image quality
metrics, e.g., by [17, 44]. However, these objective methods can still not replace
subjective assessment, but can be used as a supplement to provide a more thorough
evaluation.
Recently, [4] presented a novel, computationally efficient, iterative, spatial
gamut-mapping algorithm. The proposed algorithm offers a compromise between
90 J.-B. Thomas et al.

Fig. 4.1 Scheme of typical gamut-mapping techniques

the colorimetrically optimal gamut clipping and the most successful spatial meth-
ods. This is achieved by the iterative nature of the method. At iteration level
zero, the result is identical to gamut clipping. The more we iterate, the more we
approach an optimal, spatial, gamut-mapping result. Optimal is defined as a gamut-
mapping algorithm that preserves the hue of the image colors as well as the spatial
ratios at all scales. The results show that as few as five iterations are sufficient
to produce an output that is as good or better than that achieved in previous,
computationally more expensive, methods. Unfortunately, the method also shares
some of the minor disadvantages of other spatial gamut-mapping algorithms: halos
and desaturation of flat regions for particularly difficult images. There is therefore
much work left to be done in this direction, and one promising idea is to incorporate
knowledge of the strength of the edges.

4.3 Display Color Characterization

This section will study in depth display colorimetric characterization. Although

many books investigate color device characterization, they mostly focus on printers
or cameras, which have been far more difficult to characterize than displays
during the CRT era; thus, mostly a simple linear model and a gamma correction
were addressed in books when considering displays. With the emergence of new
technologies used to create newer displays in the last 15 years, a lot of work has
been done concerning this topic, and a new bibliography and new methods have
4 Cross-Media Color Reproduction and Display Characterization 91

Fig. 4.2 3D look-up table for a characterization process from RGB to CIELAB

appeared. Many methods have been borrowed from printers or camera though, but
the way to reproduce colors and the assumptions one can do are different when
talking about displays, so the results or the explanation of why a model is good or
not are slightly different. We propose to discuss the state of the art and the major
trends about display colorimetric characterization in this section.

4.3.1 State of the Art

Many color characterization methods or models exist; we can classify them in three
groups. In a first one, we find the models, which tend to model physically the color
response of the device. They are often based on the assumption of independence
between channels and of chromaticity constancy of primaries. Then, a combination
of the primary tristimulus at the full intensity weighted by the luminance response
of the display relatively to a digital input can be used to perform the colorimetric
transform. The second group can be called numerical models. They are based on
a training data set, which permits optimization of the parameters of a polynomial
function to establish the transform. The last category consists of 3D LUT-based
models. Some other methods can be considered as hybrid. They can be based
on a data set and assume some physical properties of the display, such as in the
work of [16].

[Link] 3D LUT Models

The models in the 3D LUT group are based on the measurement of a defined
number of color patches, i.e., we know the transformation between the input values
92 J.-B. Thomas et al.

(i.e., RGB input values to a display device) and output values (i.e., CIEXYZ or
CIELAB values) measured on the screen by a colorimeter or spectrometer in a
small number of color space locations (see Fig. 4.2). Then this transformation
is generalized to the whole space by interpolation. Studies assess that these
methods can achieve accurate results [11, 80], depending on the combination of
the interpolation method used [2,5,18,53,66], the number of patches measured, and
on their distribution [80] (note that some of the interpolation methods cited above
cannot be used with a non-regular distribution). However, to be precise enough, a lot
of measurements are typically required, i.e., a 10× 10 × 10 grid of patches measured
in [11]. Note that such a model is technology independent since no assumptions are
made about the device but that the display will always have the same response at the
measurement location. Such a model needs high storage capacity and computational
power to handle the 3D data. The computational power is usually not a problem
since Graphic Processor Units can perform this kind of task easily today [26]. The
high number of measurements needed is a greater challenge.

[Link] Numerical Models

The numerical models suppose that the transform can be approximated by a set
of equations, usually an n-order polynomial function. The parameters are retrieved
using an n-order polynomial regression process based on measurements. The
number of parameters required involves a significant number of measurements,
depending on the order of the polynomial function. The advantage of these models is
that they take into account channel interdependence by applying cross components
factors in the establishment of the function [54,55,83]. More recently, an alternative
method has been proposed by [89] who removed the three-channel crosstalk from
the model, considering that the inter-channel dependence is only due to two-channel
crosstalk, thus reducing the required number of measurements. They obtained
results as accurate as when considering the three-channel crosstalk.
Radial basis function (RBF) permits to use a sum of low-order polynomials
instead of one high-order polynomial and has been used successfully in different
works [26, 27, 79, 80]. Mostly polyharmonic splines are used, which include thin
plate splines (TPS) that [75] used for printers too. TPS are a subset of polyharmonic
splines (bi-harmonic splines). Sharma and Shaw [75] recalled the mathematical
framework and presented some applications and results for printer characterization.
They showed that using TPS, they achieved a better result than in using local
polynomial regression. They showed that by using a smoothing factor, error in
measurement impact can be avoided at the expense of the computational cost that
optimize this parameter, similar results were observed by [26]. However, [75] did
study neither data distribution influence (but they stated that the data distribution
can improve the accuracy in their conclusion) nor the use of other kernels for
interpolation. This aspect has been studied by [26], in which main improvements
were in the optimization of the selection of the data used to build the model in an
iterative way.
4 Cross-Media Color Reproduction and Display Characterization 93

[Link] Physical Models

Physical models are historically widely used for displays, since the CRT technology
follows well the assumptions cited above [13, 19, 29]. Such a model typically
first aims to linearize the intensity response of the device. This can be done by
establishing a model that assumes the response curve to follow a mathematical
function, such as a gamma law for CRT [13, 14, 28, 74], or an S-shaped curve
for LCD [58, 59, 94]. Another way to linearize the intensity response curve is to
generalize measurements by interpolation along the luminance for each primary
[68]. The measurement of the luminance can be done using a photometer. Some
approaches propose as well a visual response curve estimation, where the 50%
luminance point for each channel is determined by the user to estimate the gamma
value [28]. This method can be generalized to the retrieval of more luminance levels
in using half-toned patches [62, 65]. Recently a method to retrieve the response
curve of a projection device using an uncalibrated camera has been proposed by [8]
and extended by [62]. Note that it has been assumed that the normalized response
curve is equivalent for all the channels, and that only the gray level response curve
can be retrieved. In the case of a doubt about this assumption, it is useful to retrieve
the three response curves independently. Since visual luminance matching for the
blue channel is a harder task, it is of use to perform an intensity matching for the
red and green channel, and a chromaticity matching or gray balancing for the blue
one [57]. This method should not be used with projectors though, since they show a
large chromaticity shift with the variation of input for the pure primaries.
A model has been defined by [91, 92] for DLP projectors using a white segment
in the color wheel. In their model, the characteristics of the luminance of the white
channel is retrieved with regard to additive property of the display, given the four-
tuplet (R, G, B,W ) from an input (dr , dg , db ).
The second step of these models is commonly the use of a 3 × 3 matrix
containing primary tristimulus values at full intensity to build the colorimetric
transform from luminance to an additive independent color space. The primaries
can be estimated by measurement of the device channels at full intensity, using
a colorimeter or a spectroradiometer, assuming their chromaticity constancy. In
practice this assumption does not hold perfectly, and the model accuracy suffers
from that. The major part of the non-constancy of primaries can be corrected
by applying a black offset correction [50]. Some authors tried to minimize the
chromaticity non-constancy in finding the best chromaticity values of primaries
(optimizing the components of the 3 × 3 matrix) [30]. Depending on the accuracy
required, it is also possible to use generic primaries such as sRGB for some
applications [8], or data supplied by the manufacturer [28]. However, the use of
a simple 3 × 3 matrix for the colorimetric transform leads to inaccuracy due to
the lack of channel independence and of chromaticity constancy of primaries. An
alternative approach has been derived in the masking model and modified masking
model, which takes into account the cross-talk between channels [83]. Furthermore,
the lack of chromaticity constancy can be critical, particularly for LCD technology,
which has been shown to fail this assumption [20, 58]. The piecewise linear model
94 J.-B. Thomas et al.

assuming variation in chromaticity (PLVC) [34] is not subject to this effect, but has
not been widely used since [68] demonstrated that among the models they tested in
their article, the PLVC and the piecewise linear-assuming chromaticity constancy
(PLCC) models were of equivalent accuracy for the CRT monitors they tested. With
the last one requiring less computation, it has been more used than the former one.
These results have been confirmed in studies on CRT technology [68,69], especially
with a flare correction [50, 86]. On DLP technology when there is a flare correction,
results can be equivalent; however, PLVC can give better results on LCDs [86].
Other models exist, such as the two-steps parametric model proposed by [16].
This model assumes separation between chromaticity and intensity, and is shown to
be accurate, with average Δ Eab ∗ ’s around 1 or below for one DLP projector and a

CRT monitor. The luminance curve is retrieved, as for other physical models, but
the colorimetric transform is based on 2D interpolation in the chromaticity plane
based on a set of saturated measured colors.

[Link] The Case of Subtractive Displays

An analog film-projection system in a movie theater was studied by [3]. A Minolta

CS1000 spectrophotometer was used to find the link between the RGB colors of the
image and the displayed colors. For each device, red, green, blue, cyan, magenta,
yellow, and gray levels were measured. The low luminosity levels didn’t allow a
precise color measurement with the spectrophotometer at their disposal. For the
35 mm projector, it was found that the color synthesis is not additive, since the
projection is based on a subtractive method. It is difficult to model the transfer
function of this device; the measures cannot be reproduced as both measure and
projection angles change. Moreover, the luminance is not the same all over the
projected area. The subtractive synthesis, by removing components from the white
source, cannot provide the same color sensation as a cinema screen or a computer
screen, which is based on additive synthesis of red, green, and blue components.
Subtractive cinema projectors are not easy to characterize as the usual models are
for additive synthesis. The multiple format transformations and data compression
led to data lost and artifacts.
Ishii [49] shows the gamut differences between CRT monitors (RGB additive
method) and printed films (CMY dyes subtractive method). The main problem for a
physical modeling is the tone shift. In a matching process from a CRT to a film, both
gamut difference and mapping algorithm are important. During the production step,
the minor emulsion changes and chemical processes can vary and then make small
shifts on the prints, leading to a shift on the whole production. An implementation
of a 3D LUT was successfully applied to convert color appearance from CRT to film
display.
4 Cross-Media Color Reproduction and Display Characterization 95

4.3.2 Physical Models

[Link] Display Color Characterization Models

Physical models are easily invertible, do not require a lot of measurements, require
a little computer memory, and do not require high computing power. So, they
can be used in real time. Moreover, the assumptions of channel independence
and chromaticity constancy are appropriate for the CRT technology. However,
these assumptions (and others such as spatial uniformity, both in luminance and
in chromaticity, view angle independence, etc.) do not fit so well with some of
today’s display technologies. For instance, the colorimetric characteristic of a part
of an image in a Plasma Display is strongly dependent of what is happening in the
surrounding [25] for energy economy reasons. In LC technology, which has become
the leader for displays market, these common assumptions are not valid. Making
such assumptions can reduce drastically the accuracy of the characterization. For
instance, a review of problems faced in LC displays has been done by [94]. Within
projection systems, the large amount of flare induces a critical chromaticity shift of
primaries.
In the same time, the computing power has become less and less a problem. Some
models not used in practice because of their complexity can now be highly beneficial
for display color characterization. This section provides definitions, analysis, and
discussion about display color characterization models. We do not detail hybrid
methods or numerical methods in this section because they show less interest for
modeling purpose, and we do prefer to refer the reader to the papers cited above. 3D
LUT-based method are more considered in the part concerning model’s inversion.
In 1983, [28] wrote what is considered to be the pioneer article in the area of
physical models for display characterization. In this work, the author stated that a
power function can be used, but is not the best to fit with the luminance response
curve of a CRT device. Nevertheless, the well-known “gamma” model that considers
a power function to approximate the luminance response curve of a CRT display is
still currently widely used.
Whichever shape the model takes, the principle remains the same. First, it
estimates the luminance response of the device for each channel, using a set of
functions monotonically increasing such as (4.2). Note that the results of these
functions can also be estimated with any interpolation method, since the problem
of monotonicity that can arise during the inversion process is taken into account.
This step is followed by a colorimetric transform.

[Link] Response Curve Retrieval

We review here two types of models. The models of the first type are based
on functions, the second type is the PLCC model. This model is based on
linear interpolation of the luminance response curve and its accuracy has been
demonstrated by [68] who found it the best among the models they tested (except in
front of the PLVC model for chromatic accuracy).
96 J.-B. Thomas et al.

Fig. 4.3 Response curve in a

X, Y and Z for an LCD
display in function of the 40
digital input for, respectively, X
30
the red (a), green (b) and blue Y
(c) channel 20
Z
10

0
0 50 100 150 200 250
b
100
80
60
40
20
0
0 50 100 150 200 250
c
150

100

50

0
0 50 100 150 200 250

For function-based model, the function used is the power function for CRT
devices, which is still the most used, even if it has been shown that it does not
fit well LC technology [33]. It has been shown that for other technologies, there is
no reason to try to fit the device response with a gamma curve, especially for an
LCD technology that shows an S-shaped response curve in most cases (Fig. 4.3)
and an S-curve model can be defined [58, 59, 94]. However, the gamma function is
still often used, mainly because it is easy to estimate the response curve with a few
number of measurements, or using estimations with a visual matching pattern.
The response in luminance for a set of digital values input to the device can be
expressed as follows:
YR = fr (Dr )
YG = fg (Dg )
YB = fb (Db ), (4.2)
4 Cross-Media Color Reproduction and Display Characterization 97

where fr , fg , and fb are functions that give the YR ,YG , and YB contribution in
luminance of each primary independently for a digital input Dr , Dg , Db . Note that
for CRT devices, after normalization of the luminance and digital value, the function
can be the same for each channel. This assumption is not valid for LCD technology
[73], and is only a rough approximation for DLP-based projection systems, as seen,
for instance, in the work of [72].
For a CRT, for the channel h ∈ {r, g, b}, this function can be expressed as

YH = (ah dh + bh)γh , (4.3)

where H ∈ {R, G, B} is the equivalent luminance from a channel h ∈ {r, g, b} for a

normalized digital input dh , with dh = 2nD−1h
. Dh is the digital value input to a channel
h and n is the number of bits used to encode the information for this channel. ah is
the gain and bh is the internal offset for this channel. These parameters are estimated
empirically using a regression process.
This model is called gain-offset-gamma (GOG) [12, 48, 55]. If we make the
assumption that there is no internal offset and no gain, a = 1 and b = 0, it becomes
the simple “gamma” model.
Note that for luminance transforms, polynomials can be fitted better in the
logarithmic domain or to cube root function than in the linear domain because
the eye response to signal intensity is logarithmic (Weber’s law). For gamma-based
models, it has been shown that a second order function with two parameters such
as Log(YH ) = bh × Log(dh) + ch × (Log(dh ))2 1 gives better results[28] and that two
gamma curves should be combined for a better accuracy in low luminance[6].
For an LCD, it has been shown by [58, 59] that an S-shaped curve based on four
coefficients per channel can fit well the intensity response of the display.
α
dh h
YH = Ah × gh(dh ) = Ah × β
, (4.4)
dh h + Ch
with the same notation as above, and with Ah , αh , βh , and Ch parameters obtained
using the least-squares method. This model is called S-curve I.
The model S-curve II considers the interaction between channels. It has been
shown in [58, 59, 94] that the gradient of the original S-curve function fits the
importance of the interaction between channels. Then this component can be
included in the model in order to take this effect into account.
YR = Arr × gYRYR (dr ) + Arg × gY RYG (dg ) + Arb × gY RYB (db ),
YG = Agr × gY GYR (dr ) + Agg × gYGYG (dg ) + Agb × gY GYB (db ),
YB = Abr × gY BYR (dr ) + Abg × gY BYG (dg ) + Abb × gYBYB (db ), (4.5)

1 Note that [68] added a term to this equation, which became Log(YH ) = a + bh × Log(dh ) +
ch .(Log(dh ))2 .
98 J.-B. Thomas et al.

where g(d) and its first-order derivative g (d) are

dα (α − β )xα +β −1 + α Cxα −1
g(d) = , g (d) = . (4.6)
dβ + C (xβ + C)2

To ensure the monotonicity of the functions for the S-curve models I and II,
some constraints on the parameters have to be applied. We let the reader refer to the
discussion in the original article [59] for that matter.
For the PLCC model, the function f is approximated by a piecewise linear
interpolation between the measurements. The approximation is valid for a large
enough amount of measurements (16 measurements per channel in [68]). This
model is particularly useful when no information is available about the shape of
the display luminance response curve.

[Link] Colorimetric Transform

A colorimetric transform is then performed from the (YR ,YG ,YB ) “linearized”
luminance to the CIEXYZ color tristimulus.
⎡ ⎤ ⎡ ⎤ ⎡ ⎤
X Xr,max Xg,max Xb,max YR
⎣ Y ⎦ = ⎣ Yr,max Yg,max Yb,max ⎦ × ⎣ YG ⎦ , (4.7)
Z Zr,max Zg,max Zb,max YB
where the matrix components are the tristimulus colorimetric values of each
primary, measured at their maximum intensity.
Using such a matrix for the colorimetric transform supposes perfect additivity
and chromaticity constancy of primaries. These assumptions have been shown to be
acceptable for CRT technology [19, 29].
The channel interdependence observed in CRT technology is mainly due to an
insufficient power supply and an inaccuracy of the electron beams, which meet
inaccurately the phosphors [54]. In LC technology, it comes from the overlapping
of the spectral distribution of primaries (the color filters), and from the interferences
between the capacities of two neighboring subpixels [72, 94]. In DLP-DMD
projection devices, there is still some overlapping between primaries and inaccuracy
at the level of the DMD mirrors.
Considering the assumption of chromaticity constancy, it appears that when there
is a flare [54], either a black offset (internal flare) or an ambient flare (external flare),
added to the signal, the assumption of chromaticity constancy is not valid anymore.
Indeed, the flare is added to the output signal and the lower the luminance level of the
primaries, the more the flare is a significant fraction of the resulting stimulus. This
leads to a hue shift toward the black offset chromaticity. Often the flare has a “gray”
(nearly achromatic) chromaticity; thus, the chromaticities of the primaries shift to a
“gray” chromaticity (Fig. 4.4, left part). Note that the flare “gray” chromaticity does
not necessarily correspond to the achromatic point of the device (Fig. 4.4). In fact,
4 Cross-Media Color Reproduction and Display Characterization 99

Fig. 4.4 Chromaticity tracking of primaries with variation of intensity. The left part of the figure
shows it without black correction. On the right, one can see the result with a black correction
performed. All devices tested in our PLVC model study are shown, a-PLCD1, b-PLCD2, c-PDLP,
d-MCRT, e-MLCD1, f-MLCD2. Figures from [86]
100 J.-B. Thomas et al.

Fig. 4.4 (continued)

4 Cross-Media Color Reproduction and Display Characterization 101

in the tested LCD devices (Fig. 4.4a, b, e, f), we can notice the same effect as in
the work of [61]: the black level chromaticity is bluish because of the poor filtering
power of the blue filter in the low wavelength.
The flare can be taken all at once as the measured light for an input (dr,k , dg,k , db,k )
= (0, 0, 0) to the device. Then it includes ambient and internal flare.
The ambient flare comes from any light source reflecting on the display screen.
If the viewing conditions do not change it remains constant, can be measured and
taken into account, or can be simply removed in setting up a dark environment (note
that for a projection device, there is always an amount of light that lights the room,
coming from the bulb through the ventilation hole).
The internal flare, which is the major part of chromaticity inconstancy at least
in CRT technology [54], is coming from the black level. In CRT technology, it has
been shown that in setting the brightness to a high level, the black level increases to a
non-negligible value [54]. In LC technology, the panel let an amount of light passing
through due to a leakage of the crystal to stop all the light. In DLP technology, an
amount of light can be not absorbed by the “black absorption box,” and is focused
on the screen via the lens.
On Fig. 4.4, one can see the chromaticity shift to the flare chromaticity with the
decreasing of the input level. We have performed these measurements in a dark
room, then the ambient flare is minimized, and only the black level remains. After
black level subtraction, the chromaticity is more constant (Fig. 4.4), and a new
model can be set up in taking that into account [43, 50, 54, 55].
The gamma models reviewed above have been extended in adding an offset term.
Then the GOG can become a gain-offset-gamma-offset (GOGO) model [46,54,55].
The previous equation (4.2) becomes:

YH = (ah dh + bh)γh + c, (4.8)

where c is a term containing all the different flares in presence. If we consider the
internal offset bh as null, the model becomes gain-gamma-offset (GGO) [46].
A similar approach can be used for the PLCC model. When the black correction
[50] is performed, we name it PLCC* in the following. The colorimetric transform
used then is (4.9) that permits to take the flare into account during the colorimetric
transformation. For the S-curve models, the black offset is taken into account in the
matrix formulation in the original papers.
If we consider that mathematically, the linear transform from the linearized
RGB to CIEXYZ needs to associate the origin of RGB to the origin of CIEXYZ in
order to respect the vectorial space property of additivity and homogeneity. Thus,
the original transform of the origin of RGB to CIEXYZ needs to be translated of
[−Xk −Yk − Zk ]. However, in doing that we modify the physical reality and we need
to translate the result of the transformation of [XkYk Zk ]. We can formulate these
transforms such as in (4.9).
102 J.-B. Thomas et al.

⎡ ⎤
⎡ ⎤ ⎡ ⎤ YR
X Xr,max − Xk Xg,max − Xk Xb,max − Xk Xk ⎢ YG ⎥
⎣ Y ⎦ = ⎣ Yr,max − Yk Yg,max − Yk Yb,max − Yk Yk ⎦ × ⎢ ⎥ . (4.9)
⎣ YB ⎦
Z Zr,max − Zk Zg,max − Zk Zb,max − Zk Zk
1

The Ak ’s, A ∈ {X,Y, Z}, come from a black level estimation.

Such a correction permits the achievement of better results. However, on the
right part of Fig. 4.4, one can see that even with the black subtraction, the primary
chomaticities do not remain perfectly constant. On Fig. 4.4, right-a, it remains a
critical shift especially for the green channel.
Several explanations are involved. First, there is a technology contribution. For
LC technology, the transmittance of the cells of the panel changes within the input
voltage [20, 93]. This leads to a chromaticity shift when changing the input digital
value. For different LC displays, we notice a different shift in chromaticity; this
is due to the combination backlight/LC with the color filters. Since the filters
transmittances are optimized taking into account the transmittance shift of the LC
cells, the display can achieve good chromaticity constancy. For CRT, there are less
problems due to the same phosphors properties, as well for DLP as the light and the
filters remain the same.
However, even with the best device, there is still a small amount of non-
constancy. This leads to a discussion about the accuracy of the measured black
offset. Indeed, the measurement devices are less accurate in the low luminance.
Berns et al. [15] proposed a way to estimate the best black offset value. A way
to overcome the problems linked with remaining inaccuracy for LCD devices
has been presented by [30]. It consists in the replacement of the full intensity
measurement of primary chromaticities colorimetric values by the optimum values
in the colorimetric transformation matrix. It appears that the chromaticity shift is a
major issue for LCD. Sharma [73] stated that for LCD devices, the assumption of
chromaticity constancy was weaker than the channel interdependence.
More models that linearize the transform exist. In this section, we presented the
ones that appeared to us as the more interesting or the more known.

[Link] Piecewise Linear Model Assuming Variation in Chromaticity

Defining the piecewise linear model assuming variation in chromaticity (PLVC) in

this section has many motivations. First, it is the first display color characterization
model introduced in the literature as far as we know. Secondly, it is a hybrid method,
considering that it is based on data measurement and assumes a small amount of
hypothesis on the behavior of the display. Finally, there is a section in next chapter
devoted to the study of this model.
According to [68], the first persons who have introduced the PLVC were [34]
in 1980. Note that it preceded the well-known article from [28]. Further studies
have been performed afterward on CRT [50, 68, 69], and recently on more recent
4 Cross-Media Color Reproduction and Display Characterization 103

technologies [86]. This model does not consider the channel interdependence, but
does model the chromaticity shift of the primaries. In this section, we recall the
principles of this model, and some features that characterize it.
Knowing the tristimulus values of X, Y , and Z for each primary as a function of
the digital input, assuming additivity, the resulting color tristimulus values can be
expressed as the sum of tristimulus values for each component (i.e., primary) at the
given input level. Note that in order not to add several times the black level, it is
removed from all measurements used to define the model. Then, it is added to the
result, to return to a correct standard observer color space [50, 69]. The model is
summarized and generalized in (9.3) for N primaries, and illustrated in (4.11) for a
three primaries RGB device, following an equivalent formulation as the one given
by [50].
For an N primary device, we consider the digital input to the ith primary,
di (mi ), with i an integer ∈ [0, N], and mi an integer limited by the resolution
of the device (i.e., mi ∈ [0, 255] for a channel coded on 8 bits). Then, a color
CIEXYZ(. . . , di (mi ), . . .) can be expressed by:
i=N−1
X(. . . , di (mi ), . . .) = ∑ [X(di ( j)) − Xk ] + Xk ,
i=0, j=mi

i=N−1
Y (. . . , di (mi ), . . .) = ∑ [Y (di ( j)) − Yk ] + Yk ,
i=0, j=mi

i=N−1
Z(. . . , di (mi ), . . .) = ∑ [Z(di ( j)) − Zk ] + Zk (4.10)
i=0, j=mi

with Xk ,Yk , Zk the color tristimulus coming out from a (0, . . . , 0) input.
We illustrate this for a three primaries RGB device, with each channel coded on
8 bits. The digital input are dr (i), dg ( j), db (l), with i, j, l integers ∈ [0, 255]. In this
case, a CIEXYZ(dr (i), dg ( j), db (l)) can be expressed by:

X(dr (i), dg ( j), db (l)) = [X(dr (i)) − Xk ] + [X(dg( j)) − Xk ] + [X(db(l)) − Xk ] + Xk ,

Y (dr (i), dg ( j), db (l)) = [Y (dr (i)) − Yk ] + [Y (dg ( j)) − Yk ] + [Y (db (l)) − Yk ] + Yk ,
Z(dr (i), dg ( j), db (l)) = [Z(dr (i)) − Zk ] + [Z(dg ( j)) − Zk ] + [Z(db (l)) − Zk ] + Zk .
(4.11)

If the considered device is a RGB primaries device, thus the transformation

between digital RGB values and RGB device’s primaries is as direct as possible.
The Ak , A ∈ {X,Y, Z} are obtained by accurate measurement of the black level.
The [A(di ( j)) − Ak ], are obtained by one dimensional linear interpolation with the
measurement of a ramp along each primary. Note that any 1-D interpolation method
can be used. In the literature, the piecewise linear interpolation is mostly used.
Studies of this model have shown good results, especially on dark and mid-
luminance colors. When the colors reach higher luminance, the additivity assump-
104 J.-B. Thomas et al.

tion is less true for CRT technology. Then the accuracy decreases (depending on
the device properties). More precisely, [68, 69] stated that chromaticity error is
lower for the PLVC than for the PLCC in low luminance. This is due to the
setting of primaries colorimetric values at maximum intensity in the PLCC. Both
models show inaccuracy for high luminance colors due to channel interdependence.
Jimenez Del Barco et al. [50] found that for CRT technology, the higher level of
brightness in the settings leads to a non-negligible amount of light for a (0,0,0)
input. This light should not be added three times, and they proposed a correction
for that.2 They found that the PLVC model was more accurate in medium to high
luminance colors. Inaccuracy is more important in low luminance, due to inaccuracy
of measurements, and in high luminance, due to channel dependencies. Thomas
et al. [86] demonstrated that this model is more accurate than usual linear models
(PLCC, GOGO) for LCD technology, since it takes into account the chromaticity
shift of primaries that is a key features for characterizing this type of display. More
results for this model are presented in the next chapter.

4.4 Model Inversion

4.4.1 State of the Art

The inversion of a display color characterization model is of major importance for

color reproduction since it provides the set of digital values to input to the device in
order to display a desired color.
Among the models or methods used to achieve color characterization, we
can distinguish two categories. The first one contains models that are practically
invertible (either analytically, or in using simple 1D LUT) [13,14,29,50,54,55,68],
such as the PLCC, the black-corrected PLCC*, the GOG, or GOGO models. The
second category contains the models or methods, which are not practically invertible
directly. and that show difficulties to be applied. Models of this second category
require other methods to be inverted in practice. We can list some typical problems
and methods used to invert these models:
• Some conditions have to be verified, such as in the masking model [83].
• A new matrix might have to be defined by regression in numerical models
[54, 55, 89].
• A full optimization process has to be set up for each color, such as in S-curve
model II [58, 59] in the modified masking model, [83] or in the PLVC model
[50, 68, 84].
• The optimization process can appear only for one step of the inversion process,
as in the PLVC [68] or in the S-curve I [58, 59] models.

2 Equations (4.10) and (4.11) are based on the equation proposed by [50], and take that into account.
4 Cross-Media Color Reproduction and Display Characterization 105

• Empirical methods based on 3-D LUT (look-up table) can be inverted directly
[11], using the same geometrical structure. In order to have a better accuracy,
however, it is common to build another geometrical structure to yield the inverse
model. For instance, it is possible to build a draft model to define a new set of
color patches to be measured [80].
The computational complexity required to invert these models makes them seldom
used in practice, except the full 3-D LUT, whose major drawback is that it requires
a lot of measurements. However, these models do have the possibility to take into
account more precisely the device color-reproduction features, such as interaction
between channels or chromaticity inconstancy of the primaries. Thus, they are often
more accurate than the models of the first category.

4.4.2 Practical Inversion

Models such as the PLCC, the black-corrected PLCC*, the GOG, or GOGO models
[13, 14, 29, 50, 54, 55, 68] are easily inverted since they are based on linear algebra
and on simple functions. For these models, it is sufficient to invert the matrix of
(4.7). Then we have:
⎡ ⎤ ⎡ ⎤−1 ⎡ ⎤
YR Xr,max Xg,max Xb,max X
⎣ YG ⎦ = ⎣ Yr,max Yg,max Yb,max ⎦ × ⎣ Y ⎦ . (4.12)
YB Zr,max Zg,max Zb,max Z

Once the linearized {YR ,YG ,YB } have been retrieved, the intensity response curve
function is inverted as well to retrieve the {dr , dg , db } digital values. This task is
easy for a gamma-based model or for an interpolation-based one. However, for some
models such as the S-curve I, an optimization process can be required (note that this
response curve can be used to create a 1D LUT).

4.4.3 Indirect Inversion

When the inversion becomes more difficult, it is of use to set an optimization process
using the combination of the forward transform and the color difference (often the
euclidean distance) in a perceptually uniform color space, such as CIELAB, as cost
function. This generally leads to better results than usual linear models, depending
on the forward model, but is computationally expensive, and cannot be implemented
in real time. It is then of use to set a 3-D LUT based on the forward model. Note that
it does not mean that an optimization process is useless, since it can help to design
a good LUT.
106 J.-B. Thomas et al.

Fig. 4.5 The transform between RGB and CIELAB is not linear. Thus while using a linear
interpolation based on data regularly distributed in RGB, the accuracy is not the same everywhere
in the colorspace. This figure shows a plot of regularly distributed data in a linear space (blue dot,
left) and the resulting distribution after a cubic root transform (that mimics CIELAB transform)(red
dots, right)

Such a model is defined by the number and the distribution of the color patches
used in the LUT, and by the interpolation method used to generalize the model
to the entire space. In this subsection, we review some basic tools and methods.
We distinguish works on displays from more general works, which have been
performed in this way either in a general purpose or especially for printers. One of
the major challenges for printers is the problem of measurement, which is really
restrictive, and many works have been carried out in using a 3-D LUT for the
color characterization of these devices. Moreover, since printer devices are highly
nonlinear, their colorimetric models are complex. So it has been customary in the
last decade to use a 3-D complex LUT for the forward model, created by using
an analytical forward model, both to reduce the amount of measurements and to
perform the color space transform in a reasonable time. The first work we know
about creating a LUT based on the forward model is a patent from [81]. In this
work, the LUT is built to replace the analytical model in the forward direction. It
is based on a regular grid designed in the printer CMY color space, and the same
LUT is used in the inverse direction, simply in switching the domain and co-domain.
Note that in displays, the forward model is usually computationally simple and that
we need only to use a 3-D LUT for the inverse model. The uniform mapping of
the CMY space leads to a nonuniform mapping in CIELAB space for the inverse
direction, and it is common now to resample this space to create a new LUT. To do
that, a new grid is usually designed in CIELAB and is inverted after gamut mapping
of the points located outside the gamut of the printer. Several algorithms can be used
to redistribute the data [24, 31, 41] and to fill the grid [9, 77, 88].
4 Cross-Media Color Reproduction and Display Characterization 107

Returning to displays, let us call source space the independent color space
(typically CIELAB or alternatively CIEXYZ), the domain from where we want to
move, and destination space, the RGB color space, the co-domain, where we want
to move to. If we want to build a grid, we then have two classical approaches to
distribute the patches in the source space, using the forward model. One can use
directly a regular distribution in RGB and transform it to CIELAB using the forward
model; this approach is the same as used by [81] for printers, and leads to a non-
uniform mapping of the CIELAB space, which can lead to a lack of homogeneity
of the inverse model depending on the interpolation method used (See Fig. 4.5). An
other approach can be to distribute the patches regularly in CIELAB, following a
given pattern, such as an hexagonal structure [80] or any of the methods used in
printers [24, 31, 41]. Then, an optimization process using the forward model can be
performed for each point to find the corresponding RGB values. The main idea of
the method and the notation used in this document are the following:
• One can define a regular 3-D grid in the destination color space (RGB).
• This grid defines cubic voxels. Each one can be split into five tetrahedra (See
Fig. 4.6).
• This tetrahedral shape is preserved within the transform to the source space
(either CIEXYZ or CIELAB).
• Thus, the model can be generalized to the entire space, using tetrahedral
interpolation [53]. It is considered in this case that the color space has a linear
behavior within the tetrahedron (e.g., the tetrahedron is small enough).
The most used way to define such a grid is to take directly a linear distribution
of points on each digital dr , dg , and db axis as seeds and to fill up the rest of
the destination space. A tetrahedral structure is then built with these points. The
built structure is used to retrieve any RGB value needed to display a specific color
inside the device’s gamut. The more points are used to build the grid, the more
the tetrahedra will be small and the interpolation accurate. Each vertex is defined
by Vi, j,k = (Ri , G j , Bk ), where Ri = di , G j = d j , Bk = dk , and di , d j , dk ∈ [0, 1]
are the possible normalized digital values, for a linear distribution. i ∈ [0, Nr − 1],
j ∈ [0, Ng − 1], and k ∈ [0, Nb − 1] are the indexes (integers) of the seeds of the grid
along each primary, and Nr (resp. Nb , Ng ) is the number of steps along channel R
(resp. G, B).
Once this grid has been built, we define the tetrahedral structure for the
interpolation following [53]. Then, we use the forward model to transform the
structure into CIELAB color space. An inverse model has been built. According to
the nonlinearity of the CIELAB transform, the size of the tetrahedra is not anymore
the same as it was in RGB. In the following section, a modification of this framework
is proposed that makes this grid more homogeneous in the source color space where
we perform the interpolation; this should lead to a better accuracy, following [41].
Let us consider the PLVC model inversion as an example. This model inversion
is not as straightforward as the matrix-based models previously defined. For a three
primaries display, according to [68], it can be performed defining all subspaces
defined by the matrices of each combinations of measured data (note that the
108 J.-B. Thomas et al.

Fig. 4.6 The two ways to split a cubic voxel in 5 tetrahedra. These two methods are combined
alternatively when splitting the cubic grid to guarantee that no coplanar segments are crossing

intercepts have to be subtracted, and once all the contributions are known, they have
to be added). One can perform an optimization process for each color [50], or define
a grid in RGB, such as described above, which will allow us to perform the inversion
using 3D interpolation. Note that Post and Calhoun have proposed to define a full
LUT considering all colors. They said themselves that it is inefficient. Defining a
reduced regular grid in RGB leads to the building of an irregular grid in CIELAB
due to the nonlinear transform. This irregular grid could lead to inaccuracy or a lack
of homogeneity in interpolation, especially if it is linear. Some studies addressed
this problem [84, 85]. They built an optimized LUT, based on a customized RGB
grid.
4 Cross-Media Color Reproduction and Display Characterization 109

4.5 Quality Evaluation

Colorimetric characterization of a color display device is a major issue for the

accurate color rendering of a scene. We have seen several models that could possibly
be used for this purpose, each of these models have their own advantages and
weaknesses.
This section discusses the choice of a model in relation with the technology
and the purpose. We first address the problem of defining adequate requirements
and constraints, then we discuss the appropriate corresponding model evaluation
approach. Before concluding, we propose a qualitative comparison of some display
characterization methods.

4.5.1 Purpose

Like any image-processing technique, a display color characterization model has

to be chosen considering needs and constraints. For color reproduction, the need is
mainly the expected level of accuracy. The constraints depend mainly on two things:
the time and the measurement. The time is a major issue, because one may need
to minimize the time of establishment of a model, or its application to an image
(computational cost). The measurement process is critical because one may need
to have access to a special device to establish the model. The constraint of money
is distributed on the time, the software, and hardware cost, and particularly on the
measurement device. We do not consider here some other features of the device,
such as spatial uniformity, gamut size, etc. but only the result of the point-wise
colorimetric characterization.
In the case of displays, the combination needs vs constraints seem to be in
agreement. Let us expose two situations:
• The person who needs an accurate color characterization (such as a designer or
a color scientist) has often a color measurement device available, is working in
a more or less controlled environment, and does not mind to spend 15–20 min
every day to calibrate his/her monitor/projector. This person may typically want
to use an accurate method, an accurate measurement device, to take care of the
temporal stability of the device, etc.
• The person who wants to display some pictures in a party or in a seminar,
using a projector in an uncontrolled environment does not need a very accurate
colorimetric rendering. That is fortunate, because he/she does not have any
measurement device, does not have much time to perform a calibration or to
properly warm up the projector. However, this person needs the colors not to
betray the meaning she/he intends. In this case, a fast end-user characterization
should be precise enough. This person might use a visual calibration, or even
better, a visual/camera-based calibration. The method should be coupled to a
user-friendly software for making it easy and fast.
110 J.-B. Thomas et al.

Fig. 4.7 Evaluation of a forward model scheme. A digital value is sent to the model and to
the display. A value is computed and a value is measured. The difference between these values
represents the error of the model in a perceptually pseudo-uniform color space

We can see a duality between two types of display characterization methods and
goals: the consumer, end-user purpose, which intends only to keep the meaning and
aesthetic unchanged through the color workflow, and the accurate professional one,
which aims to have a very high colorimetric fidelity through the color workflow. We
see also through these examples that the constraints and the needs are not necessarily
going in the opposite direction.
In the next section, we will relate the quality of a model with colorimetric
objective indicators.

4.5.2 Quality

Once a model is set up, there is a need to evaluate its quality to confirm we are within
the accuracy we wanted. In this section, we discuss how to use objective indicators
for assessing quality.

[Link] Evaluation

A point-wise quality evaluation process is straightforward. We process a digital

value with the model to obtain a result and compare it in a perceptually pseudo-
uniform color space, typically CIELAB, with the measurement of the same input.
Figure 4.7 illustrates the process.
The data set used to evaluate the model should be obviously different than
the one used to yield the model. The distribution of this data can be either
distributed regularly or following a random distribution. Often, authors are choosing
an evaluation data set distributed homogeneously in the RGB device space. This is
4 Cross-Media Color Reproduction and Display Characterization 111

Table 4.1 This table shows the set of thresholds one can use to assess the
quality of a color characterization model, depending on the purpose
∗
Δ Eab Professional Consumer
Mean Δ Eab∗ Max Δ Eab∗ Mean Δ Eab∗

−<1 Good Good Good

1≤−<3 Acceptable
3≤−<6 Not acceptable Acceptable Acceptable
6≤− Not acceptable Not acceptable

a good choice, since it will cover the whole device possibility. This can also be a
good choice for the comparison of one method over different devices. However, if
one wants to relate the result to the visual interpretation of the signal throughout
the whole gamut of the device, it might be judicious to select an equiprobably
distributed data set in a perceptual color space. This means that most of the data
will fall into low digital values.

[Link] Quantitative Evaluation

Once we have an estimation of the model failure, we would like to be able to say how
it is good or not for a given purpose. The ideal colorimetric case is to have an error
below the just noticeable difference3(JND). Kang [52] stated on page 167 of his
book that the JND is of 1 Δ Eab∗ unit. Mahy et al. [60] study assessed that the JND is
∗
of 2.3 Δ Eab units. Considering that the CIELAB color space is not perfectly uniform,
it is impossible to give a perfect threshold with an euclidean metric.4 Moreover,
these thresholds have been defined for simultaneous pair comparison of uniform
color patches. This situation almost never fit with a display use, it may then not be
the best choice when comparing color display devices.
∗ thresholds for color imaging devices, many thresholds have
In the case of Δ Eab
been used [1, 42, 70, 78]. Stokes et al. [82] found a perceptibility acceptance for
pictorial images of an average of 2.15 units. Catrysse et al. [23] used a threshold of
3 units. Gibson and Fairchild [39] found acceptable a characterized display that has
a prediction error average of 1.98 and maximum of 5.57, while the non-acceptable
has at the best an average of 3.73 and a maximum of 7.63 using Δ E94 ∗ .

Following is a set of thresholds that could be used to quantify the success of

the color control depending on the purpose. In Table 4.1, we distinguish between
accurate professional color characterization, which purpose is to ensure a really high

3A JND is the smallest detectable difference between a starting and secondary level of a particular
sensory stimulus, in our case two color samples.
4 The JND while using Δ E ∗ should be closer to one than with other metrics but has still been
00
defined for simultaneous pair comparison of uniform color patches.
112 J.-B. Thomas et al.

quality color reproduction, and a consumer color reproduction, which aims only
at the preservation of the intended meaning, and relate the purpose with objective
indicators.
Considering the professional reproduction, let us consider the following rule of
thumb. If we want to reach a good accuracy, we need to consider two indicators:
the average and the maximum error. Let us consider the average: from 0 to 1 is
considered good, from 1 to 3 acceptable, and over 3 not acceptable. If now we
consider the maximum, from 0 to 3 is good, from 3 to 6 is acceptable, over is not
acceptable. If we compare this scale with the rule of thumb used by [42], it makes
sense since below three it is hardly perceptible, the same if we look at the work
of [1]. If we look at the JND proposed by [52] or [60] it seems to make sense
since in both cases, the good is under the JND. In this case we would prefer results
to be good, and it may be possible to discard a couple model/display if it does
not satisfy this condition. In the case of this professional reproduction, it could be
better to use the maximum error to discard a couple model/display. Considering the
consumer prediction, we propose to consider that from 0 to 3 it is good, from 3 to 6
it is acceptable, and over 6 it is not acceptable. In this case we would rather accept
methods that shows average results up to 6, since it should not spoil the meaning of
the reproduction. This is basically the same as the rule of thumb proposed by [42],
perceptible but acceptable being the basic idea of preserving the intended meaning.

4.5.3 Color Correction

The different approaches presented in the previous sections are characterized by

different parameters, such as the accuracy on a given technology, the computational
cost, the number of measurements required, etc.
The accuracy of the color rendering depends on the choice of both the method
and the display technology and features.
Display characteristics, such as temporal stability or spatial uniformity have to
be taken into account. Some of these parameters are studied in the literature, for
instance, in [87]. However, Table 4.2 presents a qualitative summary of different
display colorimetric characterization models based on quality thresholds from
Table 4.1, and on the experimentation on several displays of different models
in relation with the nature and number of measurements needed. The complete
quantitative analysis of these models are presented in the literature [26, 62, 86].
We only focus on five models that are a representative sampling of existing ones:
The PLVC model [50, 68, 86], Bala’s model [8, 62], an optimized polyharmonic
splines-based model [26], the offset-corrected piecewise linear assuming chromatic-
ity constancy model (PLCC*) and the GOGO [13, 14, 29, 50, 54, 55, 68].
4 Cross-Media Color Reproduction and Display Characterization 113

Table 4.2 Qualitative interpretation of different models based on Table 4.1. The efficiency of a
model is dependent on several factors: the purpose, the number of measurements, the nature of the
data to measure, the computational cost, its accuracy, etc. All these parameters depend strongly on
each display
Polyharmonic
Model PLVC Bala PLCC* splines GOGO
Type of 54 (CIEXYZ) 1–3 visual 54 (Y) 216 (CIEXYZ) 3–54 (Y)
measurement measures tasks measures 3 measures measures 3
for 1–3 (CIEXYZ) (CIEXYZ)
pictures
Technology Dependent Dependent Dependent Independent CRT
Purpose Professional or Consumer Professional or Professional Consumer
Consumer Consumer

4.6 Conclusion and Perspectives

Successful color-consistent cross-media color reproduction depends on a multitude

of factors. In this chapter we have reviewed briefly the state of the art of this field,
focusing specifically on displays.
Device colorimetric characterization is based on a model, which can successfully
predict the relationship between the media value and the color itself. A model can be
based on knowledge on the device technology, then a few measurement or evaluation
is necessary. A numerical model based on measure only can be used too, which
requires usually more measurement, and requires to take care of more aspects, such
as an interpolation method and the distribution on the training data set.
Point-wise colorimetric characterization of displays is something that is working
considering objective indicators. Within this chapter, we reviewed different means
to achieve this result. Display technology is evolving really fast. New technology
might requires the definition of other types of models. For instance, this happened
with the emergence of multi-primaries devices, which means more than three
primaries, there are some works that address the transform from a set of N-primaries
and a 3-D colorimetric data.
This chapter only treated on point-wise, static models. A research direction could
be to define dynamic models, which could take into account the spatial uniformity,
the temporal stability, etc.
Within the last section of this chapter, we mainly wanted to show how we can
evaluate the quality of a couple display/color characterization model with the tools
we have in hands and to give an idea of how to select a model for a given purpose.
In summary, the choice of a couple display/color characterization model depends
on the purpose. However, all the considerations we discussed are taking into
account colorimetric objective indicators. In the case of complex images, indicators
based on pointwise colorimetry show their limit. As far as we know, there is no
comprehensive work addressing color fidelity and quality for complex color images
on displays based on more human indicators. But there is some research initiated in
this direction.
114 J.-B. Thomas et al.

Furthermore, to reach an efficient perceived quality of displayed images, we

need to relate the work on image quality metric and the display color rendering
quality. That means to define an objective indicator for color image quality viewed
on displays related to the accuracy of the color rendering.
This point of view could be of benefit, particularly while considering new
“intelligent” displays that adapt backlight to the image content. Such displays makes
a static model inefficient.

References

1. Abrardo A, Cappellini V, Cappellini M, Mecocci A (1996) Art-works colour calibration using

the vasari scanner. In: Color imaging conference, IS&T, pp 94–97
2. Akima H (1970) A new method of interpolation and smooth curve fitting based on local
procedures. J ACM 17:589–602
3. Alleysson D, Susstrunk S (2002) Color characterisation of the digital cinema chain.
In: Research report, part II, EPFL, Switzerland
4. Alsam A, Farup I (2009) Colour gamut mapping as a constrained variational problem.
In: Salberg A-B, Hardeberg JY, Jenssen R (eds) Image analysis, 16th Scandinavian conference,
SCIA, 2009, vol 5575 of lecture notes in computer science, pp 109–117
5. Amidror I (2002) Scattered data interpolation methods for electronic imaging systems: a
survey. J Electron Imag 11(2):157–176
6. Arslan O, Pizlo Z, Allebach JP (2003) CRT calibration techniques for better accuracy including
low-luminance colors. In: Eschbach R, Marcu GG (eds) Color imaging IX: processing,
hardcopy, and applications. SPIE Proceedings, vol 5293, pp 286–297
7. Bakke AM, Farup I, Hardeberg JY (2010) Evaluation of algorithms for the determination of
color gamut boundaries. Imag Sci Tech 54(5):050,502–11
8. Bala R, Klassen RV, Braun KM (2007) Efficient and simple methods for display tone-response
characterization. J Soc Inform Disp 15(11):947–957
9. Balasubramanian R, Maltz MS (1996) Refinement of printer transformations using weighted
regression. In: Bares J (ed) Color imaging: device-independent color, color hard copy, and
graphic arts. SPIE Proceedings, vol 2658, pp 334–340
10. Barnard K, Funt B (2002) Camera characterization for color research. Color Res Appl
27(3):152–163
11. Bastani B, Cressman B, Funt B (2005) Calibrated color mapping between LCD and CRT
displays: a case study. Color Res Appl 30(6):438–447
12. Berns R (1996) Methods for characterizing CRT displays. Displays 16(4):173–182
13. Berns RS, Gorzynski ME, Motta RJ (1993) CRT colorimetry. part II: metrology. Color Res
Appl 18(5):315–325
14. Berns RS, Motta RJ, Gorzynski ME (1993) CRT colorimetry. part I: metrology. Color Res
Appl 18(5):299,314
15. Berns RS, Fernandez SR, Taplin L (2003) Estimating black-level emissions of computer-
controlled displays. Color Res Appl 28(5):379–383
16. Blondé L, Stauder J, Lee B (2009) Inverse display characterization: a two-step parametric
model for digital displays. J Soc Inform Disp 17(1):13–21
17. Bonnier N, Schmitt F, Brettel H, Berche S (2008) Evaluation of spatial gamut mapping
algorithms. In: The proceedings of the IS&T/SID fourteenth color imaging conference,
pp 56–61
18. Bookstein F (1989) Principal warps: thin-plate splines and the decomposition of deformations.
IEEE Trans Pattern Anal Mach Intell 11(6):567–585
4 Cross-Media Color Reproduction and Display Characterization 115

19. Brainard DH (1989) Calibration of a computer-controlled color monitor. Color Res Appl
14:23–34
20. Brainard DH, Pelli DG, Robson T (2002) Display characterization. Wiley, New York
21. Braun G, Ebner F, Fairchild M (1998) Color gamut mapping in a hue linearized cielab
colorspace. In: The proceedings of the IS&T/SID sixth color imaging conference: color
science, systems and applications, Springfield (VA), pp 346–350
22. Braun GJ, Fairchild MD (1999) Image lightness rescaling using sigmoidal contrast enhance-
ment functions. J Electron Imag 8(4):380
23. Catrysse PB, Wandell BA, *a PBC, W BA, B E, Gamal AE (1999) Comparative analysis of
color architectures for image sensors. In: Sampat N, Yeh T (eds) proceedings of SPIE, vol 3650,
pp 26–35
24. Chan JZ, Allebach JP, Bouman CA (1997) Sequential linear interpolation of multidimensional
functions. IEEE Trans Image Process 6:1231–1245
25. Choi SY, Luo MR, Rhodes PA, Heo EG, Choi IS (2007) Colorimetric characterization model
for plasma display panel. J Imag Sci Technol 51(4):337–347
26. Colantoni P, Thomas JB (2009) A color management process for real time color reconstruction
of multispectral images. In: Lecture notes in computer science, 16th Scandinavian conference,
SCIA, vol 5575, pp 128–137
27. Colantoni P, Stauder J, Blonde L (2005) Device and method for characterizing a colour device.
European Patent 05300165.7
28. Cowan WB (1983) An inexpensive scheme for calibration of a colour monitor in terms of CIE
standard coordinates. SIGGRAPH Comput Graph 17(3):315–321
29. Cowan W, Rowell N (1986) On the gun independency and phosphor constancy of color video
monitor. Color Res Appl 11:S34–S38
30. Day EA, Taplin L, Berns RS (2004) Colorimetric characterization of a computer-controlled
liquid crystal display. Color Res Appl 29(5):365–373
31. Dianat S, Mestha L, Mathew A (2006) Dynamic optimization algorithm for generating inverse
printer map with reduced measurements. Proceedings of international conference on acoustics,
speech and signal processing, IEEE 3
32. Dugay F, Farup I, Hardeberg JY (2008) Perceptual evaluation of color gamut mapping
algorithms. Color Res Appl 33(6):470–476
33. Fairchild M, Wyble D (1998) Colorimetric characterization of the Apple Studio display
(flat panel LCD). Munsell color science laboratory technical report
34. Farley WW, Gutmann JC (1980) Digital image processing systems and an approach to the
display of colors of specified chrominance. Technical report HFL-80-2/ONR-80, Virginia
Polytechnic Institute and State University, Blacksburg, VA
35. Farup I, Gatta C, Rizzi A (2007) A multiscale framework for spatial gamut mapping. Image
Process, IEEE Trans 16(10):2423–2435
36. Finlayson G, Hordley S, Hubel P (1998) Recovering device sensitivities with quadratic
programming. In: The proceedings of the IS&T/SID sixth color imaging conference: color
science, systems and applications, Springfield (VA): The society for imaging science and
technology, pp 90–95
37. Gatt A, Morovic J, Noriega L (2003) Colorimetric characterization of negative film for digital
cinema post-production. In: Color imaging conference, IS&T, pp 341–345
38. Gerhardt J (2007) Spectral color reproduction: model based and vector error diffusion
approaches. PhD thesis, Ecole Nationale Superieure des Telecommunications and Gjøvik
University College, URL [Link]
39. Gibson JE, Fairchild MD (2000) Colorimetric characterization of three computer displays
(LCD and CRT). Munsell color science laboratory technical report
40. Green P (ed) (2010) Color management: understanding and using ICC profiles. Wiley,
Chichester
41. Groff RE, Koditschek DE, Khargonekar PP (2000) Piecewise linear homeomorphisms: The
scalar case. In: IJCNN (3), pp 259–264
116 J.-B. Thomas et al.

42. Hardeberg J (1999) Acquisition and reproduction of colour images: colorimetric and multi-
spectral approaches. These de doctorat, Ecole Nationale Superieure des Telecommunications,
ENST, Paris, France
43. Hardeberg JY, Seime L, Skogstad T (2003) Colorimetric characterization of projection displays
using a digital colorimetric camera. In: Projection displays IX,, SPIE proceedings, vol 5002,
pp 51–61
44. Hardeberg J, Bando E, Pedersen M (2008) Evaluating colour image difference metrics for
gamut-mapped images. Coloration Technology 124:243–253
45. Hong G, Luo MR, Rhodes PA (2001) A study of digital camera colorimetric characterization
based on polynomial modeling. Color Res Appl 26(1):76–84
46. IEC:61966–3 (1999) Color measurement and management in multimedia systems and equip-
ment, part 3: equipment using CRT displays. IEC
47. International Color Consortium (2004) Image technology colour management – architecture,
profile format, and data structure. Specification ICC.1.2004–10
48. International Commission on Illumination (1996) The relationship between digital and colori-
metric data for computer-controlled CRT displays. CIE, Publ 122
49. Ishii A (2002) Color space conversion for the laser film recorder using 3-d lut. SMPTE J
16(11):525–532
50. Jimenez Del Barco L, Diaz JA, Jimenez JR, Rubino M (1995) Considerations on the calibration
of color displays assuming constant channel chromaticity. Color Res Appl 20:377–387
51. Johnson T (1996) Methods for characterizing colour scanners and digital cameras. Displays
16(4)
52. Kang HR (ed) (1997) Color technology for electronic imaging devices. SPIE Press, iSBN:
978-0819421081
53. Kasson LM, Nin SI, Plouffe W, Hafner JL (1995) Performing color space conversions with
three-dimensional linear interpolation. J Electron Imag 4(3):226–250
54. Katoh N, Deguchi T, Berns R (2001) An accurate characterization of CRT monitor
(i) verification of past studies and clarifications of gamma. Opt Rev 8(5):305–314
55. Katoh N, Deguchi T, Berns R (2001) An accurate characterization of CRT monitor (II) proposal
for an extension to CIE method and its verification. Opt Rev 8(5):397–408
56. Kimmel R, Shaked D, Elad M, Sobel I (2005) Space-dependent color gamut mapping:
A variational approach. IEEE Trans Image Process 14(6):796–803
57. Klassen R, Bala R, Klassen N (2005) Visually determining gamma for softcopy display.
In: Procedeedings of the thirteen’s color imaging conference, IS&T/SID, pp 234–238
58. Kwak Y, MacDonald L (2000) Characterisation of a desktop LCD projector. Displays
21(5):179–194
59. Kwak Y, Li C, MacDonald L (2003) Controling color of liquid-crystal displays. J Soc Inform
Disp 11(2):341–348
60. Mahy M, Eycken LVV, Oosterlinck A (1994) Evaluation of uniform color spaces developed
after the adoption of cielab and cieluv. Color Res Appl 19(2):105–121
61. Marcu GG, Chen W, Chen K, Graffagnino P, Andrade O (2001) Color characterization issues
for TFT-LCD displays. In: Color imaging: Device-independent color, color hardcopy, and
applications VII, SPIE, SPIE Proceedings, vol 4663, pp 187–198
62. Mikalsen EB, Hardeberg JY, Thomas JB (2008) Verification and extension of a camera-based
end-user calibration method for projection displays. In: CGIV, pp 575–579
63. Morovic J (2008) Color gamut mapping. Wiley, Chichester
64. Neugebauer HEJ (1937) Die theoretischen Grundlagen des Mehrfarbendruckes. Zeitschrift für
wissenschaftliche Photographie, Photophysik und Photochemie 36(4):73–89
65. Neumann A, Artusi A, Zotti G, Neumann L, Purgathofer W (2003) Interactive perception based
model for characterization of display device. In: Color imaging IX: processing, hardcopy, and
applications IX, SPIE Proc., vol 5293, pp 232–241
66. Nielson GM, Hagen H, Müller H (eds) (1997) Scientific visualization, overviews, method-
ologies, and techniques. IEEE Computer Society, see [Link]
Visualization-Overviews-Methodologies-Techniques/dp/0818677775
4 Cross-Media Color Reproduction and Display Characterization 117

67. Noriega L, Morovic J, Lempp W, MacDonald L (2001) Colour characterization of a digital

cine film scanner. In: CIC 9, IS&T / SID, pp 239–244
68. Post DL, Calhoun CS (1989) An evaluation of methods for producing desired colors on CRT
monitors. Color Res Appl 14:172–186
69. Post DL, Calhoun CS (2000) Further evaluation of methods for producing desired colors on
CRT monitors. Color Res Appl 25:90–104
70. Schläpfer K (1993) Farbmetrik in der reproduktionstechnik und im Mehrfarbendruck. In:
Schweiz SG (ed) 2. Auflage UGRA
71. Schläpfer K, Steiger W, Grönberg J (1998) Features of color management systems. UGRA
Report 113/1, Association for the promotion of research in the graphic arts industry
72. Seime L, Hardeberg JY (2003) Colorimetric characterization of LCD and DLP projection
displays. J Soc Inform Disp 11(2):349–358
73. Sharma G (2002) LCDs versus CRTs: color-calibration and gamut considerations. Proc IEEE
90(4):605–622
74. Sharma G (2003) Digital color imaging handbook. CRC Press, iSBN: 978-0849309007
75. Sharma G, Shaw MQ (2006) Thin-plate splines for printer data interpolation. In: Proc.
“European Signal Proc. Conf.”, Florence, Italy, [Link]
Eusipco2006/papers/[Link]
76. Sharma G, Trussell HJ (1996) Set theoretic estimation in color scanner characterization. J
Electron Imag 5(4):479–489
77. Shepard D (1968) A two-dimensional interpolation function for irregularly-spaced data. In:
Proceedings of the 1968 23rd ACM national conference, New York, NY, USA, pp 517–524
78. Stamm S (1981) An investigation of color tolerance. In: TAGA proceedings, TAGA proceed-
ings, pp 156–173
79. Stauder J, Colantoni P, Blonde L (2006) Device and method for characterizing a colour device.
European Patent WO/2006/094914, EP1701555
80. Stauder J, Thollot J, Colantoni P, Tremeau A (2007) Device, system and method for
characterizing a colour device. European Patent WO/2007/116077, EP1845703
81. Stokes M (1997) Method and system for analytic generation of multi-dimensional color lookup
tables. United States Patent 5612902
82. Stokes M, Fairchild MD, Berns RS (1992) Precision requirements for digital color reproduc-
tion. ACM Trans Graph 11(4):406–422, DOI [Link]
83. Tamura N, Tsumura N, Miyake Y (2003) Masking model for accurate colorimetric characteri-
zation of LCD. J Soc Inform Disp 11(2):333–339
84. Thomas JB, Colantoni P, Hardeberg JY, Foucherot I, Gouton P (2008) A geometrical approach
for inverting display color-characterization models. J Soc Inform Disp 16(10):1021–1031
85. Thomas JB, Colantoni P, Hardeberg JY, Foucherot I, Gouton P (2008) An inverse display color
characterization model based on an optimized geometrical structure. In: Color imaging XIII:
processing, hardcopy, and applications, SPIE, SPIE proceedings, vol 6807, pp 68, 070A–1–12
86. Thomas JB, Hardeberg JY, Foucherot I, Gouton P (2008) The PLVC color characterization
model revisited. Color Res Appl 33(6):449–460
87. Thomas JB, Bakke AM, Gerhardt J (2010) Spatial nonuniformity of color features in projection
displays: a quantitative analysis. J Imag Sci Technol 54(3):030,403
88. Viassolo DE, Dianat SA, Mestha LK, Wang YR (2003) Practical algorithm for the inversion of
an experimental input-output color map for color correction. Opt Eng 42(3):625–631
89. Wen S, Wu R (2006) Two-primary crosstalk model for characterizing liquid crystal displays.
Color Res Appl 31(2):102–108
90. Wyble DR, Berns RS (2000) A critical review of spectral models applied to binary color
printing. Color Res Appl 25:4–19
91. Wyble DR, Rosen MR (2004) Color management of DLP projectors. In: Proceedings of the
twelfth color imaging conference, IS&T, pp 228–232
92. Wyble DR, Zhang H (2003) Colorimetric characterization model for DLP projectors. In:
Procedeedings of the eleventh color imaging conference, IS&T, pp 346–350
93. Yeh P, Gu C (1999) Optics of liquid crystal display. Wiley, New York, ISBN: 978-0471182016
118 J.-B. Thomas et al.

94. Yoshida Y, Yamamoto Y (2002) Color calibration of LCDs. In: Tenth color imaging con-
ference, IS&T - The society for imaging science and technology, pp 305–311, Scottsdale,
Arizona, USA
95. Yule JAC, Nielsen WJ (1951) The penetration of light into paper and its effect on halftone
reproductions. In: TAGA Proceedings, vol 3, p 65
Chapter 5
Dihedral Color Filtering

Reiner Lenz, Vasileios Zografos, and Martin Solli

The color is a body of flesh where a heart beats

Malcolm de Chazal

Abstract Linear filter systems are used in low-level image processing to analyze
the visual properties of small image patches. We show first how to use the theory
of group representations to construct filter systems that are both steerable and are
minimum mean squared error solutions. The underlying groups are the dihedral
groups and the permutation groups and the resulting filter systems define a transform
which has many properties in common with the well-known discrete Fourier
transform. We also show that the theory of extreme value distributions provides
a framework to investigate the statistical properties of the vast majority of the
computed filter responses. These distributions are completely characterized by only
three parameters and in applications involving huge numbers of such distributions,
they provide very compact and efficient descriptors of the visual properties of the
images. We compare these descriptors with more conventional methods based on
histograms and show how they can be used for re-ranking (finding typical images in
a class of images) and classification.

Keywords Linear filtering • Low-level image processing • Theory of group

representations • Dihedral groups • Permutation groups • Discrete Fourier trans-
form • Image re-ranking and classification

R. Lenz () • M. Solli

Department of Science and Technology, Linköping University, Linköping, Sweden
e-mail: [Link]@[Link]; [Link]@[Link]
V. Zografos
Computer Vision Laboratory, Linköping University, Linköping, Sweden
e-mail: zografos@[Link]

C. Fernandez-Maloigne (ed.), Advanced Color Image Processing and Analysis, 119

DOI 10.1007/978-1-4419-6190-7 5,
© Springer Science+Business Media New York 2013
120 R. Lenz et al.

5.1 Introduction

Linear filter systems are used in low-level image processing to analyze the visual
properties of small image patches. The patches are analyzed by computing the
similarity between the patch and a number of fixed filter functions. These similarity
values are used as descriptors of the analyzed patch. The most important step in this
approach is the selection of the filter functions. Two popular methods to select them
are the minimum mean squared error (MMSE) criterion and the invariance/steerable
filter approach. The MMSE method, like the jpeg transform coding, selects those
filters that allow a reconstruction of the original patch with a minimal statistical
error. The invariance, or the more general steerable, filter system assumes that
the patches of interest come in different variations and that one fixed selection of
filters can detect both, if a given patch is of interest and if this is the case which
variant of these patches it is. A typical example is an edge detector that is used to
detect if a given patch was an edge and in the case of an edge it gives a possible
estimate of its orientation (see [4, 15] for a comparison of different types of local
descriptors). For the case of digital color images, we will show how the theory of
group representations can be used to construct such steerable filter systems and that,
under fairly general conditions, these filters systems are of the MMSE type.
The general results from representation theory show that the filter functions
implement a transform which has many properties in common with the well-known
discrete Fourier transform. One of these properties that is of interest in practical
applications is the existence of fast transforms. As is the case for the Fourier
transform where the DFT can be computed efficiently by using the FFT, it can be
shown here that the basic color filter operations can be optimized by computing
intermediate results. Apart of the speedup achievable by reducing the number of
necessary arithmetic operations, we will also see that the bulk of the computations
are simple additions and subtractions which make these filters suitable for hardware
implementations and applications where a huge number of images have to be
processed. A typical example of such a task is image retrieval from huge image
databases. Such databases can contain millions or billions of images that have to
be indexed and often it is also necessary to retrieve images from such a database at
very high speed. We will therefore illustrate some properties of these filter systems
by investigating properties of image databases harvested from websites.
In our illustrations, we will show that the theory of extreme value distributions
provides a framework to investigate the statistical properties of the vast majority
of the computed filter responses. These distributions are completely characterized
by only three parameters and in applications involving huge numbers of such
distributions, they provide very compact and efficient descriptors of the visual
properties of the images. We will also compare these descriptors with more
conventional methods based on histograms and show how they can be used for re-
ranking (finding typical images in a class of images) and classification.
5 Dihedral Color Filtering 121

5.2 Basic Group Theory

In the following, we will only consider digital color images defined on a square grid.
This is the most important case in practice, but similar results can be derived for
images on hexagonal grids. We also assume that the pixel values are RGB vectors.
Our first goal is to identify natural transformations that modify spatial configurations
of RGB vectors.
We recall that a group is a set of elements with a combination rule that maps a
pair of elements to another element in the set. The combination rule is associative,
every element has an inverse and there is a neutral element. More information about
group theory can be found in every algebra textbook and especially in [3]. In the
following, we will only deal with dihedral groups. Such a group is defined as the
set of all transformations that map a regular polygon into itself. In the following,
we will use the dihedral group D4 of the symmetry transformations of the square to
describe the transformation rules of the grid on which the images are defined. We
will also use the group D3 formed by the symmetry transformations of a triangle
to describe the modifications of RGB vectors. A description of the usage of the
dihedral groups to model the geometric properties of the sensor grid can be found
in [8, 9]. This was then extended to the investigation of color images in [10, 11].
Here we will not describe the application of the same ideas in the study of RGB
histograms which can be found in [13].
It can be shown that the dihedral group Dn of the n-sided regular polygon consists
of 2n elements. The symmetry group D4 of the square grid has eight elements: the
four rotations ρk with rotation angles k = kπ /2, k = 0, . . . , 3 and the reflection σ
on one of the diagonals combined with one of the four rotations. The elements are
thus given by ρ0 , . . . , ρ3 , ρ0 σ , . . . , ρ3 σ . This is a general property of the dihedral
groups where we find for the hexagonal grid the corresponding transformation
group D6 consisting of twelve elements, six rotations and six rotations combined
with a reflection. For the RGB vectors, we consider the R, G, and B channel as
represented by corner points of an equilateral triangle. The symmetry group of the
RGB space is thus the group D3 . It has six elements ρ0 , ρ1 , ρ2 , ρ0 σ , ρ1 σ , ρ2 σ where
the ρk now represent rotations with rotation angle k · 120◦ and σ is the reflection on
one fixed symmetry axis of the triangle. Algebraically, this group is identical to the
permutation group S(3) of three elements.
We introduce the following notation to describe the general situation: For a
group G with elements g and a set Z with elements z we say that G operates on Z
if there is a mapping (G, Z) → Z such that (g, z) → gz and - - (g2 g1 )z = g2 (g1 z); we
also say that G is a transformation group. Furthermore, -Z- denotes the number of
- -
elements in the set and -G- is the number of the group elements. In the context of
the dihedral groups the set Z consists -of the - n corner points of the regular polygon,
the transformation group is Dn and -Dn - = 2n. In the following, we need more
general point configurations than the corner points of the polygon and therefore
we introduce X as a set of points in the 2D-plane and we will use x for its elements.
Such a set X is the collection of points on which the filter functions are defined. As a
122 R. Lenz et al.

second set we introduce Y consisting of the three points of an equilateral triangle.

Sometimes we write y for a general element, yi for a given element and R, G, or B if
we consider the three points as locations of the R, G, or B channel of a color image.
For the elements in the dihedral groups, we use g or gi to denote elements in D4
and h or hi for elements in D3 . For a point x ∈ X on the grid and an element g ∈ D4 ,
we define the point gx as the point obtained by applying the transformation g on the
point x. In the same way, we define for an element h and an RGB vector y = (R, G, B)
the permuted RGB vector by hy.
We now combine the spatial transformations of the grid and the permutation
of the color channels defining the product of groups. The pairs (g, h) form another
group D4 ⊗ D3 under componentwise composition ((g1 , h1 ) (g2 , h2 ) = (g1 g2 , h1 h2 )).
Finally,
. we /define the orbit D4 x of a point x under the grid operations as the
set gx, g ∈ D4 . It is easy to show that there exist three types of orbits: the one-point
orbit consisting of the origin x = (0, 0), four-point orbits,
. and eight-point
/ orbits.
Four-point orbits consist of the corner points of squares (±n, ±n) .
For finite groups, we can use a matrix–vector notation to describe the action of
the abstract transformation group. As an example, consider the case of the RGB
vectors and the group D3 . Here we identify the R-channel with the first, the G-
channel with the second, and the B-channel with the third unit vector in a three-
dimensional space. The 120◦ rotation ρ maps the RGB vector to the vector GBR
and the corresponding matrix h(ρ ) describing the transformation of the 3D unit
vectors is given by the permutation matrix:
⎛ ⎞
001
h(ρ ) = ⎝ 1 0 0 ⎠ .
010

Using this construction, we mapped abstract group elements to matrices in such

a way that concatenation of group operations corresponds to the multiplication of
matrices. General transformation groups are of limited use in practice since sets
have no internal structure. The connection between an abstract group operation
and a matrix is an example of a more interesting construction that is obtained
when the set is replaced by a vector space. A general method to construct such a
vector space as in our application as follows: We define an RGB patch p as a linear
mapping X ⊗ Y → R where p(x, y) is the value at pixel x in channel y. RGB patches
form a vector space P. The elements in D4 ⊗ D3 operate on the domain of p and we
can therefore define the transformed patch p(g,h) as p(g,h) (x, y) = p(g−1 x, h−1 y).
For example, if g is a 90◦ rotation and e is the identity operation in D3 then
the patch p(g,e) is the rotated original patch and if e is the identity in D4 and h
interchanges the R and the G channel, leaving B fixed the p(e,h) is the pattern with
the same geometrical configuration
- - but with R and G channel interchanged. The
pattern space P has dimension 3-X- where a basis is given by the set of all functions
that have value one for exactly one combination (x, y) and is zero everywhere.
5 Dihedral Color Filtering 123

The mapping p → p(g,h) is a linear mapping of pattern space and we can therefore
describe it by a matrix T(g, h) such that p(g,h) = T(g, h)p. We calculate:

p(g1 g2 ,h1 h2 ) (x, y) = p((g1 g2 )−1 x, (h1 h2 )−1 y) = p(g−1 −1 −1 −1

2 g1 x, h2 h1 y)
 (g1 ,h1 )
= p(g2 ,h2 ) (g−1 −1
1 x, h1 y) = p
(g2 ,h2 )
(x, y)

and we see that the matrices satisfy T(g1 g2 , h1 h2 ) = T(g1 , h1 )T(g2 , h2 ). This shows
that this rule defines a (matrix) representation of the group which is a mapping from
the group into a space of matrices such that the group operation maps to matrix
multiplication.
Matrices describe linear mappings between vector spaces in a given coordinate
system. Changing the basis in the vector space gives a new description of the
same linear transformations by different matrices. Changing the basis in the pattern
space P using a matrix B will replace the representation matrices T(g, h) by
the matrices BT(g, h)B−1 . It is therefore natural to construct matrices B that
simplify the matrices BT(g, h)B−1 for all group elements (g, h) simultaneously. The
following theorem from the representation theory of finite groups collects the basic
results that give a complete overview over the relevant properties of these reduced
matrices (see [3, 19, 20] for details):
Theorem 1. – We can always find a matrix B such that all BT(g, h)B−1 are block-
diagonal with blocks Tm of minimum size.
(4) (3)
– These smallest blocks are of the form T(g, h) = Ti (g) ⊗ T j (h) where ⊗
denotes the Kronecker product of matrices.
(4) (3)
– The dimensions of Ti (g) and T j (h) are one or two.
(4) (3)
– Both, the Ti and the T j , are representations and from their transformation
properties follows that it is sufficient to know them for one rotation and the
reflection: T(.) (ρ k σ l ) = (T(.) (ρ ))k (T(.) (σ ))l ).
For the group D3 operating on RGB vectors, it is easy to see that the trans-
formation (R, G, B) → R + G + B is a projection on a one-dimensional subspace
invariant under all elements in D3 . The first block of the matrices is therefore
one-dimensional and we have T(3) (h) = 1 for all h ∈ D3 . This defines the trivial
representation of the group and the one-dimensional subspace of the RGB space
defined by this transformation property is the space of all gray value vectors. The
other block is two-dimensional and given by the orthogonal complement to this
one-dimensional invariant subspace. This two-dimensional complement defines the
space of complementary colors. For the group D4 , a complete list of its smallest
representations can be found in Table 5.1.
Given an arbitrary set X closed under D4 , the tools from the representation
theory of finite groups provide algorithms to construct the matrix B such that the
transformed matrices BT(g, h)B−1 are block-diagonal with minimum-sized blocks.
For details, we refer again to the literature [3, 9, 19, 20].
124 R. Lenz et al.

Table 5.1 Representations

Name T (4) (σ ) T (4) (ρ ) Space Dimension
of D4
Trivial 1 1 Vt 1
Alternating −1 −1 Va 1
p 1 −1 Vp 1
m −1
  1  Vm 1
1 0 0 1
2D V2 2
0 −1 −1 0

Fig. 5.1 4 × 4 pattern

Practically we construct the filter functions as follows: We start with a set of

points X which is the union of D4 −orbits. The number of orbits is denoted by K
and from the orbit with
0
index k, we select an arbitrary but fixed representative xk .
We thus have X = k D4 xk . We denote the number of four-point - - orbits by K4 , of
eight-point orbits by K8 . The number of points in X is thus -X- = 4K4 + 8K8 + K0
where K0 = 1 if the origin is in X and zero otherwise. The orbit number k defines
a subspace Pk of pattern space P of dimension 3, 12, or 24 depending on the
number of points in the orbit. These spaces Pn are then further decomposed into
tensor products of the spaces in Table 5.1 and the one-dimensional subspace of the
intensities and the two-dimensional space of complementary colors.
As an illustration consider the case of a four-by-four window consisting of 16
grid points and a 48-dimensional pattern space. We construct the spatial filters by
first splitting the 16 points in the 4 × 4 window into two four-point (green cross and
blue disks) and one eight-point orbit (red squares) as shown in Fig. 5.1. The four-
point orbit splits into the direct sum Vt ⊕ Va ⊕ V2 . The eight point orbit is the direct
sum Vt ⊕Va ⊕Vp ⊕Vm ⊕ 2V2, where 2V2 denotes two copies of V2 . Combining these
decompositions reveals that the 16-dimensional space V is given by the direct sum

V = 3Vt ⊕ 3Va ⊕ Vp ⊕ Vm ⊕ 4V2. (5.1)

Next, we use the tensor representation construction and find for the structure of the
full 48-dimensional space the decomposition

V = (3Vti ⊕ 3Vai ⊕ Vpi ⊕ Vmi ⊕ 4V2i)

⊕ (3Vtc ⊕ 3Vac ⊕ Vpc ⊕ Vmc ⊕ 4V2c) , (5.2)
5 Dihedral Color Filtering 125

Table 5.2 Tensor Space Dimension

representations
Vti ,Vai ,Vpi ,Vmi 1
V2i ,Vt2 ,Va2 ,Vp2 ,Vm2 2
V22 4

where Vxy = Vx ⊗ Vy is the vector space defined by the tensor representation Tx ⊗ Ty

of the representation Tx , x = t, a, p, m, 2 of the group D4 and the representation Ty ,
y = t, 2 of D3 . The dimensions of the representation spaces are collected in Table 5.2.

5.3 Illustration

We will now illustrate some properties of these filter systems with the help of a
single image. We use an image of size 192 × 128 and filters of size 4 × 4. The
properties of the filter results are of course depending on the relation between
the resolution of the original image and the filter size where images with a high
resolution will on average contain more homogeneous regions. We selected the
combination of a small image size and small filter size since we will later use this
combination in an application where we will investigate the statistical properties of
databases consisting of very many thumbnails harvested by an image search engine
from the internet.
In Fig. 5.2, we see the original image with the two parrots in the upper left corner
and the result of the 48 different filters. The first 16 filter results are computed from
the intensity channel and the remaining from the 32 color-opponent combinations
corresponding to the splitting of the original 48-dimensional vector space described
in Table 5.2. The images show the magnitude of the filter responses. In the case
of the intensity-based filters, this means that a pattern and its inverted copy will
produce the same filter result in this figure. The colormap used in Fig. 5.2 and the
following figures is shown below the filter results. From the figure, we can see that
the highest filter responses are obtained by the filters that are related to the spatial
averaging, i.e., those belonging to vector spaces Vti and Vti . We see also that the
intensity filters have in general higher responses than their corresponding color-
opponent filters. From the construction (see (5.2)), we saw that these 48 filters
come in 24 packages of length one, two, and four where all filters in a package have
the same transformation properties and the norm of the filter vectors is invariant
under spatial and color transformations. The norm of these 24 filter packages is
shown in Fig. 5.3. Again, we see the highest response for the three spatial averaging
filters (besides the original) and the three spatial averaging filters combined with
the color-opponent colors (last two in the third row and first in the fourth row).
In both, Figs. 5.2 and 5.3 the filter results are scaled such that the highest value for
all filter responses is one. This makes it possible to see the relative importance of
the different filters and filter packages. In Fig. 5.4, we show the magnitude of the
126 R. Lenz et al.

Fig. 5.2 Original image and the 48 filter results

Fig. 5.3 Original image and the 24 magnitude filter images

5 Dihedral Color Filtering 127

Fig. 5.4 Selected line and edge filters. Dark corresponds to large magnitude and light corresponds
to low magnitude

filter response vectors for four typical filter packages related to the vector spaces of
types Vai ,V2i ,Vac , and V2c . Visually, they correspond to line and edge filters in the
intensity and the color-opponent images.

5.4 Linear Filters

Up to now we described and classified the different transformations of the pattern

space. Now we will consider functions on these patterns and here especially
functions that define projections. We define a linear filter f as a linear map from
the pattern space P into the real (or complex) scalars. The Riesz representation
theorem [22] shows that this map is represented
1 2 by an element in pattern space and
that 1we have: f : p → R; p →
f (p) = f , p = f  p where f  is the transposed vector
2
and f , p is the scalar product. An L-dimensional linear filter system F is an L-tupel
of linear filters. We write
1 2 1 2 1 2
F(p) = F, p = f1 , p , . . . , fL , p .

We call such a vector a feature vector and the corresponding vector space the feature
space.
128 R. Lenz et al.

We now divide the pattern space P into the smallest subspaces under the
transformation group introduced above. It is then easy to see that the filter systems
defined as the projection operators onto these invariant subspaces have two prop-
erties that are of importance in applications: invariance and steerability. Consider
a filter system F that is a projection on such an invariant subspace. From the
construction, we know that a pattern p in this subspace can be described by a coor-
dinate vector F(p) and since the subspace is invariant we find for the transformed
pattern p(g,h) a transformation matrix T(g, h) such that F(p(g,h) ) = T(g, h)F(p).
From the general theory, it can also be shown that we can always choose 3 the 3
coordinate system such that the matrices T(g, h) are orthonormal. The norm 3F(p)3
of the feature vector is thus invariant under all group transformations. Due to the
symmetry of the scalar product we can also apply the transformations of the group to
the filter functions and thus modify their behavior. The feature vectors obtained from
the modified filter systems are also related via the matrix multiplication with T(g, h)
and we see that we can generate all possible modifications of these feature vectors
from any given instance of it. A formal description is the following: a filter system
is steerable if it satisfies the following condition:
1 2 1 2 1 2
4 h) F, p for an L × L matrix T(g,
F, p(g,h) = F, T(g, h)p = T(g, 4 h).

If we collect all filter coefficients in the matrix F, then a steerable filter system
4 h)F.
satisfies the matrix equations FT(g, h) = T(g,
For a fixed pattern p0 , we can generate its orbit (D4 ⊗ D3 )p0 in pattern space
4 h)F(p0 ) define
and if F is a steerable filter system then the feature vectors T(g,
an orbit1 in feature
2 space. Steerable filters have the advantage that the actual filter
vector F, p0 is computed once all the transformed versions can be computed with
4 h)F.
the closed form expression T(g,
In summary, we constructed filter systems F with the following properties:

4 h)F(p) for all transformations (g, h) ∈ D4 ⊗ D3 and the filter

– F(p(g,h) ) = T(g,
system is covariant.
3 3 3 3
– 3F(p(g,h) )3 = 3F(p)3 for all transformations (g, h) ∈ D4 ⊗ D3 , i.e. the norm of
the feature vector is an invariant.
– The filter system is steerable via T(g, h)F(p).
– The filter system is irreducible, i.e., the length of the feature vector is minimal.
– Two filter systems that are of the same type, i.e., belonging to the same
product Tx ⊗ Ty of irreducible representations, have identical transformation
properties under the transformations in the group. Especially all feature values
computed by the same type of filter systems, applied to different orbits, obey
identical transformation rules.
5 Dihedral Color Filtering 129

5.5 Statistical Properties

5.5.1 Partial Principal Component Analysis

Up to now we only used the existence of transformation groups to design filter

systems. More powerful results can be obtained if we also make assumptions about
the statistical properties of the transformations. A reasonable assumption, especially
for spatially small patches, is that all transformations are equally likely to be
encountered in a sufficiently large set of measurements. In this case, we can compute
the second-order moments by first summing up over all transformed versions of a
given pattern and then over different patterns. Formally, we consider the patterns p
as output of a stochastic process with variable ω : p = pω , ω ∈ Ω . The second-

order moment matrix, or correlation matrix, C is proportional to C = ∑ω pω pω .
For a general group G with elements g and transformed patterns pg = T(g)p, we
 
find for the correlation matrix C = ∑Ω pω pω = ∑G ∑Ω /G T(g)pω pω T(g) . Here,
we assumed that all transformations g ∈ G have the same probability and we
denote by Ω /G the equivalence classes of elements in Ω that are not related via
a group transformation. From this follows that the correlation matrix satisfies the
equations C = T(g)CT(g) for all g ∈ G. For an orthonormal matrix T(g), this
implies CT(g) = T(g)C and a matrix C with this property is called an intertwining
operator. If the matrices T(g) are those defined by the smallest invariant subspaces
then Schur’s lemma [3, 7, 19, 20] states that the matrix C must either be the null-
matrix or a scalar multiple of the unit matrix. From this it follows that in the
coordinate system based on the smallest invariant subspaces, a general correlation
matrix with C = T(g)CT(g) for all g ∈ G must be block-diagonal. The number of
blocks is given by the number of different types of invariant subspaces.
Under the condition that all transformations in the transformation group are
equally likely, the matrix of second-order moments is block-diagonal. In reality,
this will never be exactly the case and for every collection of images we therefore
have to compute how good the block-diagonal approximation describes the current
collection. As an example we use the image collections [Link] and DPChallenge
described in [2]. From the [Link] database, we used twenty three million patches
from 18,365 images and from DPChallenge 16,508 images and twenty million
patches. We resized the images so that the smallest dimension (height or width) was
128 pixels. The second-order moment matrix computed from the resized images is
shown in Fig. 5.5. From the colorbar, it can be seen that all entries in the matrix
have approximately the same value but one can also see that the matrix has a certain
geometrical structure. For the second-order moment matrix computed from the
filtered data, we find that the contributions from the values from the first, averaging,
filters are much higher than the contributions from the other filters. We therefore
illustrate the properties of these filter magnitudes in logarithmic coordinates.
In Fig. 5.6, we see the logarithms of the diagonal elements of the matrix of
second-order moments. The vertical lines mark the boundary between the different
blocks given by the dihedral structure. The first blocks (components 1–16) are
130 R. Lenz et al.

Fig. 5.5 Second-order moment matrix computed from original data

computed from the intensity channel, whereas the remaining (17–48) are related
to the two-dimensional complementary color distributions. The first block (1–3 in
intensity, 17–22 in complementary color) is computed by spatial averaging, the
second (4–6 and 23–28) to line-like structures, the third (7–10, 29–44) to edges,
and the last (11–12, 45–48) is related to the one-dimensional representations p and
m (see (5.1)) from the inner orbit. The left diagram shows the values computed from
the [Link] images, the right diagram is based on the DPChallenge images. We
see that the non-negativity of the first filter results leads to significant correlations
with the other filter results. This leads to the structures in the first column and the
last rows in the figures in the left column. The structure in the remaining part of the
matrices can be clearly seen in the two images in the right column.
In Fig. 5.7, the structure of the full second-order moment matrices of the filtered
results is shown. On the left side of the figure the full matrices are shown; in the
right column, the rows and columns related to the averanging filters are removed to
enhance the visibility of the structure in the remaining part. In the upper row, the
results for the [Link] and in the lower row the DPChallenge database were used.

5.5.2 Extreme Value Theory

We may further explore the statistical properties of the filters with the help of the
following simple model: consider a black-box unit U with input X the pixel values
from a finite-sized window in a digital image (a similar analogy can be applied to
the receptive fields of a biological vision system). The purpose of this black box is to
5 Dihedral Color Filtering 131

Fig. 5.6 Log Diagonal of the second-order moment matrices after filtering (a) [Link]
(b) DPChallenge

measure the amount of some non-negative quantity X(t) that changes over 5
time. We
write this as u(t) = U(X(t)). We also define an accumulator s(n) = 0n u(t)dt that
accumulates the measured output from the unit until it reaches a certain threshold
s(n) = Max(n) (X) or a certain period of time, above which the accumulator is reset
to zero and the process is restarted.
If we consider u(t), s(n) as stochastic processes and select a finite number N
of random samples u1 , . . . , uN , then their joint distribution J(u1 , . . . , uN ) and the
distribution Y (sN ) of sN , depend on the original distribution F(XN ). At this point,
we may pose two questions:
1. When N → ∞ is there a limiting form of Y (s) → Φ (s)?
2. If there exists such a limit distribution, what are the properties of the black-box
unit U and of J(u1 , . . . , uN ) that determines the form of Φ (s)?
132 R. Lenz et al.

Fig. 5.7 Second-order moment matrices (a) Full Matrix [Link] (b) No averaging filters
[Link] (c) Full Matrix DPChallenge (d) No averaging filters DPChallenge

In [1], the authors have demonstrated that under certain conditions on Y (s) the
possible limiting forms of Φ (s) are given by the three distribution families:
  
μ −s
Φ (s) = exp − exp , ∀s Gumbel,
σ
)  *
s−μ k
Φ (s) = 1 − exp − , s > μ Weibull,
σ
)   *
s − μ −k
Φ (s) = exp − , s > μ Fréchet, (5.3)
σ

where μ , σ , k are the location, scale, and shape parameters of the distributions,
respectively. The particular choice between the three families in (5.3) is determined
by the tail behavior of F(X) at large X. In this case, we use as units U the black
box that computes the absolute value of the filter result vectors from the irreducible
5 Dihedral Color Filtering 133

Fig. 5.8 Image type and model distribution in EVT parameter space

representations of the dihedral groups. The filter vectors not associated with the
trivial representation are of the form s = ∑(xi − x j ) where xi , x j are pixel values.
We can therefore expect that these filter values are usually very small and that high
values will appear very seldom. In addition, these sums are calculated over a small,
finite neighborhood, and for this reason, the random variables are highly correlated.
In short, the output for each filter has a form similar to the sums described in [1],
and so it is possible to use the EVT to model their distribution.
We may now analyze which types of images are assigned to each submodel
in (5.3). For economy of space, we only illustrate a single filter (an intensity
edge filter) on the dataset described in Sect. 5.7.1, but the results generalize to all
filters and different datasets. We omit the μ parameter since it usually exhibits
very little variation and the most important behavior is observed in the other two
parameters. First of all, if we look at Fig. 5.8 we see a correlated dispersion in the
two axes, with the Fréchet images spanning only a very small region of the space
at low σ , k, and well separated from 2-parameter and 3-parameter Weibull. Also
notice how the Fréchet set typically includes images with near-uniform colored
regions with smooth transitions between them, or alternatively very coarse-textured,
homogeneous regions with sharp boundaries. High frequency textures seem to be
relatively absent from the Fréchet, and on average the image intensities seem to be
lower in the Fréchet set than in the Weibulls.
On the other hand, the 2-parameter and 3-parameter Weibull clusters are
intermixed, with the 2-parameter mostly restricted to the lower portion of the space.
For smaller σ , k values, the 2-parameter Weibull images exhibit coarser textures,
with the latter becoming more fine-grained as σ , k increase in tandem. Also, there
134 R. Lenz et al.

Fig. 5.9 A comparison between the extrema and other regions of a filtered image (a) Original
image (b) Edge filter result (c) Tails (maxima) (d) Mode (e) Median (f) Synthesis

seems to be a shift from low-exposure, low-contrast images with shadows (small

σ , k) to high-contrast, more illumination, less shadows when σ , k become large.
Furthermore, the 2-parameter Weibull set shows a preference for sharp linear edges
associated with urban, artificial, or man-made scenes, whereas the 3-parameter
Weibull mostly captures the “fractal”-type edges, common in nature images.
Finally, we illustrate the importance of the data at the extrema of a filtered image,
as described by the EVT. In Fig. 5.9a, we show an image (rescaled for comparison)
and its filtered result using one of the intensity edge filters in Fig. 5.9b. This is
essentially a gradient filter in the x- and y-directions. Next is Fig. 5.9c that shows
the response at the tails of the fitted distribution. It is immediately obvious that the
tails contain all the important edges and boundary outlines that abstract the main
objects in the image (house, roof, horizon, diagonal road). These are some (but not
necessarily all) of the most salient features that a human observer may focus on,
or that a computer vision system might extract for object recognition or navigation.
We also show the regions near the mode in Fig. 5.9d. We see that much of it contains
small magnitude edges and noise from the almost uniform sky texture. Although
this is part of the scene, it has very little significance when one is trying to classify
or recognize objects in an image. A similar observation holds for the grass area,
which although contains stronger edges than the sky and is distributed near the
median (Fig. 5.9e), it is still not as important (magnitude-wise and semantically)
as the edges in the tails are. Finally, Fig. 5.9f shows how all the components put
together, can describe different regions in the image: the salient object edges in the
tails (red); the average response, discounting extreme outliers, (median) in yellow;
the most common response in light blue (mode); and the remaining superfluous
data in between (dark blue). This is exactly the type of semantic behavior that the
EVT models can isolate with their location, scale, and shape parameters, something
which is not immediately possible when using histograms.
5 Dihedral Color Filtering 135

5.6 Fast Transforms, Orientation, and Scale

From the construction of the filters follows that they can be implemented as a
combination of three basic transforms: one operating on the RGB vectors, one
for the four-point, and one for the eight-point orbit. These filters are linear and
they are therefore completely characterized by three matrices of sizes 3 × 3, 4 ×
4, and 8 × 8. The rows of these matrices define projection operators onto the
spaces Vxy introduced above and they can be computed using algorithms from the
representation theory of finite groups.
We illustrate the basic idea with the help of the transformation matrix related
to the RGB transformation. We already noted that the sum R+G+B is invariant  
under permutations of the RGB channels. It follows that the vector 1 1 1
defines a projection onto this invariant subspace. We also know that the orthogonal
complement to this one-dimensional subspace defines a two-dimensional
  subspace
that cannot be reduced further. Any two vectors orthogonal to 1 1 1 can therefore
be used to fill the remaining two rows of the RGB transformation matrix. Among the
possible choices we mention here two: the Fourier transform and an integer-valued
transform. The Fourier transform is a natural choice considering the interpretation of
the RGB vector as three points on a triangle. In this case, the remaining two matrix
rows are given by cos(2kπ /3) and sin(2kπ /3). Since the filters are applied to a large
number of vectors, it is important to find implementations
 that
 are computationally
 
efficient. One solution is given by the two vectors 1 −1 0 and 1 1 −2 . The
resulting transformation matrix has the advantage that the filters can be implemented
using addition and subtraction only. We can furthermore reduce the number of
operations by computing intermediate sums. One solution is to compute RG =
R+G first and combine that afterward to obtain RG+B and RG-2B. The complete
transformation can therefore be computed with the help of five operations instead
of the six operations required by a direct implementation of the matrix–vector
operation.
For the four- and the eight-point orbit transforms, we can use the general tools of
representation theory to find the integer-valued transform matrices:
⎛ ⎞
1 1 1 1 1 1 1 1
⎜ 1 1 −1 −1 1 1 −1 −1 ⎟
⎛ ⎞ ⎜ ⎟
⎜ 1 −1 1 −1 1 −1 1 −1 ⎟
1 1 1 1 ⎜ ⎟
⎜ −1 1 −1 1 ⎟ ⎜ ⎟
⎜ ⎟ ⎜ −1 1 1 −1 −1 1 1 −1 ⎟
⎝ −1 −1 1 1 ⎠ and ⎜ ⎟. (5.4)
⎜ 0 −1 −1 0 0 1 1 0 ⎟
⎜ ⎟
1 −1 −1 1 ⎜ 1 0 0 −1 −1 0 0 1 ⎟
⎜ ⎟
⎝ −1 0 0 −1 1 0 0 1 ⎠
0 1 −1 0 0 −1 1 0

Also here we see that we can use intermediate sums to reduce the number of
operations necessary. This is an example of a general method to construct fast-group
theoretical transforms similar to the FFT implementation of the Fourier transform.
More information can be found in [18].
136 R. Lenz et al.

One of the characteristic properties of these filter systems is their covariance

property described above, i.e., F(p(g,h) ) = T(g, 4 h)F(p) for all transformations
(g, h) ∈ D4 ⊗ D3 . In Fig. 5.3, we used only the fact that the matrix T(g, 4 h) is
orthonormal and the norm of the filter vectors is therefore preserved when the
underlying pixel distribution undergoes one of the transformations described by
the group elements. More detailed results can be obtained by analyzing the
transformation properties of the computed feature vectors. The easiest example of
how this can be done is related to the transformation
 of the filter result computed
from a 2 × 2 patch with the help of the filter −1 1 −1 1 in the second row of the
matrix in (5.4). Denoting the pixels intensity values in the 2 × 2 patch by a, b, c, d,
we get the filter result F = (b + d) − (a + c). Rotating the patch 90◦, gives the pixel
distribution d, a, b, c with filter value (a + c) − (d + b) = −F. In the same way, we
find that a reflection on the diagonal gives the new pixel vector a, d, c, b with filter
result F = (d + b) − (a + c). Since these two operations generate all possible spatial
transforms in D4 , we see that the sign change of the filter result indicates if the
original patch was rotated 90◦ or 270◦ . We can therefore add the sign of the filter
results as another descriptor. Using the same calculations, we see that reflections
and 180◦ rotations cannot be distinguished from the original.
This example can be generalized using group theoretical tools as follows: In the
space of all filter vectors F (in the previous case, the real line R) one introduces
an equivalence relation defining two vectors F1 , F2 as equivalent if there is an
element (g, h) ∈ D4 ⊗ D3 such that F2 = T(g, 4 h)F1 . In the previous case, the
equivalence classes are represented by the half-axis. Every equivalence class is
given by an orbit of a given feature vector and in the general case, there are up
to 48 different elements (corresponding to the 48 group elements) in such an orbit.
Apart from the norm of the feature vector, we can thus characterize every feature
vector by its position on its orbit relative to an arbitrary but fixed element on
the orbit. This can be used to construct SIFT-like descriptors [14] where the bins
of the histogram are naturally given by the orbit positions. Another illustration
is related to the two-dimensional representation V2i representing intensity edge
filters. Under the transformation of the original distribution, the computed feature
vectors transform in the same way as the dihedral group transforms the points of
the square. From this follows directly that the magnitude of the resulting two-
dimensional filter vector (Fx , Fy ) is invariant under rotations and reflections and
represents edge-strength. The relation between the vector components (Fx , Fy ) is
related to orientation and we can therefore describe (Fx , Fy ) in polar coordinates
as vector (ρ , θ ). In Fig. 5.10, this is illustrated for the edge filters computed from
the inner 2 × 2 corner points. The magnitude image on the left corresponds to the
result shown in Fig. 5.4 while the right images encodes the magnitude ρ in the v-
component and the angular component θ in the h-component of the hsv-color space.
The results described so far are all derived within the framework of the dihedral
groups describing the spatial and color transformation of image patches. Another
common transform which is not covered by the framework is scaling. Its systematic
analysis lies outside the scope of this description but we mention one strategy that
can be used to incorporate scaling. The basic observation is that the different orbits
5 Dihedral Color Filtering 137

Fig. 5.10 Edge magnitude (left) and orientation (right)

Fig. 5.11 Scale-space

under the group D4 are related to scaling. In the simplest case of the three averaging
intensity filters, one gets as filter results the vector (F1 , F2 , F3 ) representing the
average intensity over the three different rings in the 4 × 4 patch. Assuming that
the scaling is such that one can, on average, interchange all three orbits then one
can treat the three filter results F1 , F2 , F3 as function values defined on the corners
of a triangle. In that case, one can use the same strategy as for the RGB components
and apply the D3 -based filters to the vectors (F1 , F2 , F3 ). The first filter will then
compute the average value over three different scales. Its visual effect is a blurring.
The second filter computes the difference between the intensities in the inner four-
pixel patch and the intensities on the eight points on the next orbit. The visual
property is a center-surround filter. Finally, the third filter computes the difference
between the two inner and the outer orbit. Also, here we can convert the vectors with
the last two filter results to polar coordinates to obtain a magnitude “blob-detector”
and an angular phase-like result. We use the same color coding as for the edge-
filter illustration in Fig. 5.10 and show the result of these three scale-based filters
in Fig. 5.11. We don’t describe this construction in detail but we only show its
effect on the values of the diagonal elements in the second-order moment matrix. In
Fig. 5.12, we show the logarithms of the absolute values of the diagonal elements
in the second-order moment matrix computed from the filtered patches as before
(marked by crosses) and the result of the scaling operation (given by the circles).
For the first three filters, we see that the value of the first component increased
138 R. Lenz et al.

Fig. 5.12 Diagonal elements and scaling operation

significantly while the values of the other two decreased correspondingly. This is a
typical effect that can also be observed for the other filter packages. We conclude
the descriptions of these generalizations by remarking that this is an illustration
showing that one can use the representation theory of the group D4 ⊗ D3 ⊗ D3 to
incorporate scaling properties into the framework.

5.7 Image Classification Experiments

Among possible applications that can be based on the presented filter systems we
will here illustrate the usefulness in an image classification experiment, where we
try to separate classes of images downloaded from the Internet. A popular approach
for large-scale image classification is to combine global or local image histograms
with a supervised learning algorithm. Here we derive a 16 bins histogram for each
filter package, resulting in a 16 × 24 representation of each image. The learning
algorithm is the Support Vector Machine implementation SVMlight described in [6].
For simplicity and reproducibility reasons, all experiments are carried out with
default settings.

5.7.1 Image Collections

Two-class classification results are illustrated for the keyword pairs garden–
beach and andy warhol–claude monet. We believe these pairs to be representa-
tive examples of various tasks that can be encountered in image classification.
5 Dihedral Color Filtering 139

Table 5.3 Two-class classification accuracy for various filter packages, and the overall descriptor
Filter package
Keyword pair 1:3 4:6 7:10 13:15 16:18 19:22 ALL ALL+EVT
Garden–beach 0.78 0.77 0.79 0.69 0.68 0.67 0.80 0.79
Andy warhol–claude monet 0.66 0.81 0.81 0.80 0.82 0.83 0.89 0.84

The Picsearch1 image search service is queried with each keyword, and 500
thumbnail images (maximum size 128 pixels) are saved from each search result.
Based on recorded user statistics, we only save the most popular images in each
category, which we assume will increase the relevance of each class. The popularity
estimate is based on the ratio between how many times an image has been clicked
and viewed in the public search interface. For each classification task, we create
a training set containing every second image from both keywords in the pair, and
remaining images are used for evaluating the classifier.

5.7.2 Classification Results

Classification results are given in Table 5.3. The overall result for the entire
descriptor (the entire 16 × 24 representation) is shown in the second to last column,
and classification results in earlier columns are based on selected filter packages.
The last column summarizes the classification results obtained from the EVT-
parameter descriptions of the distributions. The classification accuracy is given by
the proportion of correctly labeled images. A value of 0.75 means that 75% of the
images were labeled with a correct label. We conclude that the best classification
result is obtained when the entire descriptor is used. But as the table indicates, the
importance of different filter packages varies with the image categories in use. We
see, for instance, that the color content of an image (captured in filter packages
13–24) is more important for the andy warhol–claude monet classification result,
than for garden–beach.
We illustrate the classification result by plotting subsets of classified images. The
result based on the entire descriptor, for each keyword pair respectively, can be seen
in Fig. 5.13a, b. Each sub-figure shows the 10+10 images than obtained the most
positive and most negative score from the Support Vector Machine. Similar plots
for selected filter packages are shown in Figs. 5.14–5.19.
In closing, we briefly illustrate the practical application of the Extreme Value
theory models in the above classification examples and as an alternative represen-
tation to histograms. The input data vector to the SVM in this case contains the
three parameters: location, scale, shape estimated by fitting the EVT models to each

1 [Link]
140 R. Lenz et al.

Fig. 5.13 Classification examples based on the entire descriptor (filter results 1–24) (a) beach
(top) vs garden (bottom) (b) andy warhol (top) vs claude monet (bottom)

Fig. 5.14 Classification examples based on filter package: 1–3 (intensity mean) (a) beach (top) vs
garden (bottom) (b) andy warhol (top) vs claude monet (bottom)

of the 24 filters packages. Compared with the histogram from before, we are now
only using a 3 × 24-dimensional vector for the full filter descriptor, as opposed to a
16 × 24-dimensional vector. This leads to a much reduced data representation, faster
training and classification steps, and no need to optimally set the number of bins.
First, in Fig. 5.20 we show the comparative results for a single filter classification
on the andy warhol–claude monet set. We can see that the EVT, even with its lower
dimensionality, is equally or sometimes even more accurate than the histogram
representations. In terms of absolute accuracy numbers, the EVT scores for the full-
filter descriptor are shown in the last column of Table 5.3. As it is obvious, these
scores are very close to the histogram-based results.
5 Dihedral Color Filtering 141

Fig. 5.15 Classification examples based on filter package: 4–6 (intensity lines) (a) beach (top) vs
garden (bottom) (b) andy warhol (top) vs claude monet (bottom)

Fig. 5.16 Classification examples based on filter package: 7–10 (intensity edges) (a) beach (top)
vs garden (bottom) (b) andy warhol (top) vs claude monet (bottom)

Finally, we show an example of image retrieval. More specifically, using the

full 3 × 24-dimensional EVT-based filter descriptor we classified the four classes
of 500 thumbnail images each. We trained an SVM with 70% of the images and
tested on the remaining 30% using the “One-to-All” classification scheme to build a
multi-way classifier. The top SVM-ranked images resulting from this classification
(or in other words, the top retrieved images) are shown in Fig. 5.21.
Although the results need not be identical to those using the histograms above,
we can see the equally good retrieval and separation in the four different classes. In
particular, the vivid, near-constant colors and sharp edges in the Warhol set and the
less saturated, softer tones, and faint edges of the Monet set. In the same way, the
garden images contain very high frequency natural textures and the beach images
142 R. Lenz et al.

Fig. 5.17 Classification examples based on filter package: 13–15 (color mean) (a) beach (top) vs
garden (bottom) (b) andy warhol (top) vs claude monet (bottom)

Fig. 5.18 Classification examples based on filter package: 16–18 (color lines) (a) beach (top) vs
garden (bottom) (b) andy warhol (top) vs claude monet (bottom)

more homogeneous regions with similarly colored boundaries. These characteristics

are the exact information captured by the filters and the EVT models and which can
be used very effectively for image classification and retrieval purposes.

5.8 Summary

We started from the obvious observations that the pixels of digital images are located
on grids and that, on average, the three color channels are interchangeable. These
two properties motivated the application of tools from the representation theory of
5 Dihedral Color Filtering 143

Fig. 5.19 Classification examples based on filter package: 19–22 (color edges) (a) beach (top) vs
garden (bottom) (b) andy warhol (top) vs claude monet (bottom)

Fig. 5.20 Two-class classification accuracy comparison between the EVT and the histogram
representations for the andy warhol–claude monet set. We only use a single filter paclage at a
time

finite groups and we showed that in this framework, we can explain how steerable
filter systems and MMSE-based transform-coding methods are all linked to those
group theoretical symmetry properties. Apart from these theoretical properties,
the representation theory also provides algorithms that can be used to construct the
filter coefficients automatically and it also shows how to create fast filter systems
using the same principles as the FFT-implementations of the DFT. We also sketched
briefly how the group structure can be used to define natural bins for the histogram
descriptors of orientation parameters. A generalization that includes simple scaling
properties was also sketched.
144 R. Lenz et al.

Fig. 5.21 Retrieval results from the four classes using EVT (a) beach (top) vs garden (bottom)
(b) andy warhol (top) vs claude monet (bottom)

The computational efficiency of these filter systems makes them interesting

candidates in applications where huge numbers of images have to be processed at
high speed. A typical example of such an application is image database retrieval
where billions of images have to be analyzed, indexed, and stored. For image
collections, we showed that the statistical distributions of the computed filter values
can be approximated by the three-types of extreme value distributions. This results
in a descriptor where the statistical distribution of a feature value in an image can be
at most three parameters. We used the histograms of the filter results and the three
parameters obtained by a statistical parameter estimation procedure to discriminate
web images from different image categories. These tests show that most of the
feature distributions can indeed by described by the three-parameter extreme-value
distribution model and that the classification performance of these parametric model
descriptors is comparable with conventional histogram-based descriptors. We also
illustrated the visual significance of the distribution types and typical parameter
vectors with the help of the scatter plot in Fig. 5.8.
We did not discuss if these filter systems are relevant for the understanding of
biological vision systems and we did not compare them in detail with other filter
systems like those described in [16]. Their simple structure, their optimality prop-
erties (see also [5, 17]), and the fact that they are massively parallel should motivate
their further study in situations where very fast decisions are necessary [21]. Finally,
we mention that similar techniques can be used to process color histograms [13] and
that there are similar algorithms for data defined on three-dimensional grids [12].

Acknowledgements The financial support of the Swedish Science Foundation is gratefully

acknowledged. The research leading to these results has received funding from the European
Community’s Seventh Framework Programme FP7/2007–2013—Challenge 2—Cognitive Sys-
tems, Interaction, Robotics—under grant agreement No 247947-GARNICS.
5 Dihedral Color Filtering 145

References

1. Bertin E, Clusel M (2006) Generalised extreme value statistics and sum of correlated variables.
J Phys A: Math Gen 39(24):7607–7619
2. Datta R, Li J, Wang JZ (2008) Algorithmic inferencing of aesthetics and emotion in natural
images: An exposition. In: 2008 IEEE International Conference on Image Processing, ICIP
2008, pp 105–108, San Diego, CA
3. Fässler A, Stiefel EL (1992) Group theoretical methods and their applications. Birkhäuser,
Boston
4. Freeman WT, Adelson EH (1991) The design and use of steerable filters. IEEE Trans Pattern
Anal Mach Intell 13(9):891–906
5. Hubel DH (1988) Eye, brain, and vision. Scientific American Library, New York
6. Joachims T (1999) Making large-scale support vector machine learning practical. MIT Press,
Cambridge, pp 169–184
7. Lenz R (1990) Group theoretical methods in image processing. Lecture notes in computer
science (Vol. 413). Springer, Heidelberg
8. Lenz R (1993) Using representations of the dihedral groups in the design of early vision filters.
In: Proceedings of international conference on acoustics, speech, and signal processing, pp
V165–V168. IEEE
9. Lenz R (1995) Investigation of receptive fields using representations of dihedral groups. J Vis
Comm Image Represent 6(3):209–227
10. Lenz R, Bui TH, Takase K (2005) Fast low-level filter systems for multispectral color images.
In: Nieves JL, Hernandez-Andres J (eds) Proceedings of 10th congress of the international
colour association, vol 1, pp 535–538. International color association
11. Lenz R, Bui TH, Takase K (2005) A group theoretical toolbox for color image operators. In:
Proceedings of ICIP 05, pp III–557–III–560. IEEE, September 2005
12. Lenz R, Carmona PL (2009) Octahedral transforms for 3-d image processing. IEEE Trans
Image Process 18(12):2618–2628
13. Lenz R, Carmona PL (2010) Hierarchical s(3)-coding of rgb histograms. In: Ranchordas A
et al (eds) Selected papers from VISAPP 2009, vol 68 of Communications in computer and
information science. Springer, Berlin, pp 188–200
14. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis
60(2):91–110
15. Mikolajczyk K, Schmid C (2005) A performance evaluation of local descriptors. IEEE Trans
Pattern Anal Mach Intell 27(10):1615–1630
16. Oliva A, Torralba A (2006) Building the Gist of a Scene: The Role of Global Image Features
in Recognition. Progress in Brain Research 155:23-36
17. Olshausen BA, Field DJ (1996) Emergence of simple-cell receptive field properties by learning
a sparse code for natural images. Nature 381(6583):607–609
18. Rockmore D (2004) Recent progress and applications in group FFTs. In: Byrnes J, Ostheimer
G (eds) Computational noncommutative algebra and applications. Kluwer, Dordrecht
19. Serre J-P (1977) Linear representations of finite groups. Springer, New York
20. Terras A (1999) Fourier analysis on finite groups and applications. Cambridge University
Press, Cambridge
21. Thorpe S, Fize D, Marlot C (1996) Speed of processing in the human visual system. Nature
381(6582):520–522
22. Yosida K (1980) Functional analysis. Springer, Berlin
Chapter 6
Color Representation and Processes
with Clifford Algebra

Philippe Carré and Michel Berthier

The color is stronger than the language

Marie-Laure Bernadac

Abstract In the literature, colour information of pixels of an image has been

represented by different structures. Recently algebraic entities such as quaternions
or Clifford algebras have been used to perform image processing for example.
We propose to review several contributions for colour image processing by using the
Quaternion algebra and the Clifford algebra. First, we illustrate how this formalism
can be used to define colour alterations with algebraic operations. We generalise
linear filtering algorithms already defined with quaternions and review a Clifford
color edge detector. Clifford algebras appear to be an efficient mathematical tool
to investigate the geometry of nD images. It has been shown for instance how to
use quaternions for colour edge detection or to define an hypercomplex Fourier
transform. The aim of the second part of this chapter is to present an example of
applications, namely the Clifford Fourier transform of Clifford algebras to colour
image processing.

Keywords Clifford • Quaternion • Fourier transform • Algebraic operations •

Spatial filtering

P. Carré ()
Laboratory XLIM-SIC, UMR CNRS 7252, University of Poitiers, France
e-mail: [Link]@[Link]
M. Berthier
Laboratory MIA (Mathématiques, Images et Applications), University of La Rochelle, France
e-mail: [Link]@[Link]

C. Fernandez-Maloigne (ed.), Advanced Color Image Processing and Analysis, 147

DOI 10.1007/978-1-4419-6190-7 6,
© Springer Science+Business Media New York 2013
148 P. Carré and M. Berthier

6.1 Introduction

Nowadays, as multimedia devices and internet are becoming accessible to more

and more people, image processing must take colour information into account
because colour processings are needed everywhere for new technologies. In this
idea, several approaches have been submitted to deal with colour images, one of the
oldest is to process any greyscale algorithm on each channel of the colour image
to get the equivalent result. Implementing such programs often creates artefacts,
so researchers have come to deal differently with colour image information.
A quite recent manner to process colour algorithms is to encode the three channel
components on the three imaginary parts of a quaternion as proposed by S.T.
Sangwine and T. Ell in [1–3]. Recently algebraic entities such as Clifford algebras
have been used to perform image processing. We propose in this chapter to review
several contributions for colour image processing by using the Quaternion algebra
and the Clifford algebra.
The first section describes the spatial approach and illustrates how this formalism
can be used to define colour alterations with algebraic operations. By taking
this approach, linear filtering algorithms can be generalized with Quaternion and
Clifford algebra. To illustrate what can be done, we will conclude this part by present
strategies for colour edge detector.
The second part introduces how to use quaternions to define an hypercomplex
Fourier transform, and after reviews a Clifford Fourier transform that is suitable
for colour image spectral analysis. There have been many attempts to define such a
transformation using quaternions or Clifford algebras. We focus here on a geometric
approach using group actions. The idea is to generalize the usual definition based on
the characters of abelian groups by considering group morphisms from R2 to spinor
groups Spin(3) and Spin(4). The transformation is parameterized by a bivector and
a quadratic form, the choice of which is related to the application to be treated.

6.2 Spatial Approach

In this first section, we start with a brief description of basic concepts of Quaternion
and Clifford algebra.

6.2.1 Quaternion Concept

[Link] Definition of Quaternion

A quaternion q ∈ H is an extension of complex numbers with multiplication rules

as follows: i2 = j2 = k2 = −1, k = i j = − ji, i = jk = −k j, and j = ki = −ik and is
defined as q = qr + qi i + q j j + qk k where:
6 Color Representation and Processes with Clifford Algebra 149

• qr , qi , q j , and qk are real numbers.

• i, j, and k are two new imaginary numbers, asserting:

i2 = j2 = k2 = −1, i j = − ji = k, jk = −k j = i, ki = −ik = j. (6.1)

We see that the quaternion product is anti-commutative.

With q = qr + qi i + q j j + qk k any quaternion, we review a similar vocabulary:
• q = qr − qi i − q j j − qk k is q’s conjugate.
• If q = 0, then q−1 = |q|q 2 is q’s inverse.
• ℜ(q) = qr is q’s real part. If ℜ(q) = q, then q is real.
• ℑ(q) = ib + jc + kd is q’s imaginary part. If ℑ(q) = q, then q is pure.
√
• q’s modulus or norm is qr + qi + q j + qk = qq noted |q|.
2 2 2 2

• P = {q ∈ H | q = ℑ(q)} is the Pure Quaternion group.

• S = {q ∈ H | |q| = 1} is the Unitary Quaternion group.
As we have said Quaternion can be expressed in a scalar S(q) an vector part V (q),

q = S(q) + V(q)

with S(q) = qr and V (q) = qi i + q j j + qk k.

[Link] R3 Transformations with Quaternions

As pure quaternion are used analogously to R3 vectors, the classical R3 transforma-

tions: translations, reflections, projections, rejections, and rotations can be defined
with only additions and multiplications.
With (q1 , q2 ) ∈ P2 two pure quaternions, the translation vector is supported by
the quaternion qtrans = q1 + q2 . If q ∈ P and μ ∈ S ∩ P then qrefl = − μ qμ is the q’s
reflection vector with μ axis.
If q ∈ P and μ ∈ S ∩ P then qproj = 12 (q − μ qμ ) is the q’s projection vector on μ
axis.
If q ∈ P and μ ∈ S ∩ P then qrej = 12 (q + μ qμ ) is the q’s orthogonal projection
vector on μ axis’s orthogonal plan or the q’s rejection of the μ axis.
φ φ
And if q ∈ P, φ ∈ R and μ ∈ S∩P then qrot = eμ 2 qe−μ 2 is the q’s rotation vector
around μ axis with φ angle.
All these definitions of transformation allowed us the understand how to interact
on colour encoded in quaternion using just operations like additions, substractions
and multiplications. Furthermore Sangwine used them to create his spatial quater-
nionic filters
Sangwine and Ell were the first to use this vector part of quaternion to encode
colour images. They took the three imaginary parts to code the colour components
r (red), g (green), and b (blue) of an image. A colour image is then considered as a
map f : R2 −→ H0 defined by
150 P. Carré and M. Berthier

Fig. 6.1 Hue, saturation, and

value given with reference
μgrey and H(ν ) = 0

f [m, n] = r[m, n]i + g[m, n] j + b[m, n]k,

where r(x, y), g(x, y), and b(x, y) are the red, green, and blue components of the
image.
From a colour described in RGB colour space with a quaternion vector q ∈ P,
HSV colour space coordinates can be found as well with operations on quaternions.
We consider that Value is the norm of the colour’s orthogonal projection vector
(q.μgrey )μgrey on the grey axis μgrey (this axis can be defined such that μgrey = i+√j+k
3
.
Saturation and Hue are represented on the orthogonal plan to the grey axis which
crosses (q.μgrey )μgrey . The Saturation is the distance between the colour vector q
and the grey axis μgrey , and Hue is the angle between the colour vector q and a
colour vector ν taken anywhere on the plan orthogonal to μgrey and which sets the
reference zero Hue angle. This reference Hue value is often taken to represent the
red colour vector, so we decided arbitrarily to associate the red colour vector or any
other one to the ν vector and gave it a zero Hue value (Fig. 6.1). Hue is the angle
between this reference colour vector and the colour vector q.
If q is a colour vector, then Value V , Saturation S, and Hue H can be given with
the grey-axis μgrey ∈ S∩P and the reference colour vector ν ∈ S∩P with elementary
quaternionic operations as below
⎧ μν qν μ |
⎪
⎪ H=tan−1 |q−
⎪
⎪ |q−ν qν |
⎨
S=| 12 (q + μ qμ )| . (6.2)
⎪
⎪
⎪
⎪
⎩ V =| 12 (q − μ qμ )|

[Link] Quaternionic Filtering

After a first view of several possible manipulations on colour image encoded with
quaternions, we focus on applications which can be done with. We propose to study
low level operations on colour images like filtering operations for instance.
6 Color Representation and Processes with Clifford Algebra 151

Fig. 6.2 Sangwine edge

detector scheme

We propose to review the quaternionic colour gradient defined by Sangwine

based on the possibility to use quaternion to represent R3 transformations and
applied to colour geometry.
As a lot of gradient detector, which are of first order, the following method uses
convolution filters. Sangwine proposed the convolution on a colour image can be
defined by:
n1 m1
qfiltered (s,t) = ∑ ∑ hl (τ1 , τ2 )q((s − τ1 )(t − τ2 ))hr (τ1 , τ2 ), (6.3)
τ1 =−n1 τ2 =−m1

where hl and hr are the two conjugate filters of dimension N1 × M1 where N1 =

2n1 + 1 ∈ N and M1 = 2m1 + 1 ∈ N.
From this definition of the convolution product, Sangwine proposed a colour
edge detector in [2]. In this method, the two filters h1 and h2 are conjugated in order
to fulfill a rotation operation of every pixel around the greyscale axis by an angle of
π and compare it to its neighbours (Fig. 6.2).
The filter composed by a pair of quaternion conjugated filters is defined as follow
⎡ ⎤ ⎡ ⎤
1 1 1 1 1 1
1⎣ 1
hl = 0 0 0 ⎦ and hr = ⎣ 0 0 0 ⎦ , (6.4)
6 6
QQQ QQQ
π
where Q = eμ 2 and μ = μgrey = i+√j+k3
the greyscale axis.
The filtered image is a greyscale image almost everywhere, because the vector
sum of one pixel to its neighbours rotated by π around the grey axis has a low
saturation (q3 and q4 ). However, pixels in colour opposition (q1 and q2 , for example,
representing a colour edge) present a vector sum far from the grey axis, so edges
are coloured due to this high distance (Fig. 6.3).
After this description of some basic issues of colour processing by using
Quaternion, we introduce the second mathematical tool: Clifford algebra.
152 P. Carré and M. Berthier

Fig. 6.3 Sangwine edge

detector result

6.2.2 Clifford Algebra

[Link] Definition

The Clifford algebra framework allows to encode geometric transformations via al-
gebraic formulas. Let us fix an orthonormal basis (e1 , e2 , e3 ) of the vector space R3 .
We embed this space in a larger 8-dimensional vector space, denoted R3,0 , with
basis given by a unit 1, the three vectors e1 , e2 , e3 and the formal products e1 e2 ,
e2 e3 , e1 e3 , and e1 e2 e3 . The key point is that elements of R3,0 can be multiplied: the
product of ei and e j is, for example, ei e j . The rules of multiplication are given by:

e2i = 1, ei e j = −e j ei .

The geometric product of two vectors u and v of R3 is given by the formula

uv = u · v + u ∧ v, (6.5)

where · denotes the scalar product and u ∧ v is the bivector generated by u and v.
Since the ei ’s are orthogonal, then

ei e j = ei ∧ e j .

The linear combinations of the elements ei ∧ e j are called grade-2 entities termed
bivectors. They encode pieces of two-dimensional vector subspaces of R3 with
a magnitude and an orientation. In these algebras, multivectors which are the
extension of vectors to higher dimensions, are the basic elements. One example
is that we often represent vectors as one-dimensional directed quantities (also
represented by arrows), they are thus represented in geometric algebras by 1-vectors.
As their dimension is one, they are said 1-graded. By extension, in geometric
6 Color Representation and Processes with Clifford Algebra 153

algebra, there are grade-2 entities termed bivectors which are plane segments
endowed with orientation. In general, a k-dimensional oriented entity is known as a
k-vector. For an overview on geometric algebras see [4–7] for instance. In geometric
algebra, oriented subspaces are basic elements, as vectors in a m-dimensional linear
vector space V m . These oriented subspaces are called blades, and the term k-blade
is used to describe a k-dimensional homogeneous subspace. A multivector is then a
linear combination of blades.
Any multivector M1 ∈ Rn,0 is so described by the following equation:

n
M= ∑ Mk (6.6)
k=0

with Mk the k-vector part of any multivector M i.e., the grade k operator.
The Geometric product is an associative law and distributive over the addition of
multivectors. In general, the result of the geometric product is a multivector. As we
have said, if used on 1-vectors a and b, this is the sum of the inner and outer product:
ab = a.b + a ∧ b. Note also that the geometric product is not commutative.
This product is used to construct k-dimensional subspace elements from inde-
pendent combinations of blades. For example, multiplying with this product the two
independent 1-vectors e1 and e2 gets the bi-vector e12 . And if you multiply again
this bivector by the third 1-vector in V 3 , e3 , you get the trivector e123 . The basis of
R3,0 algebra is so given by (e0 , e1 , e2 , e3 , e23 , e31 , e12 , e123 ). Here e0 stands for the
element of grade 0, it is so the scalar part.
• External or Wedge product: The wedge product is denoted by ∧.
Wedge product can then be described using geometric product as follow with
A a s-graded multivector and B a r-graded multivector, both in Rn,0 :

Ar ∧ Bs = Ar Bs r+s . (6.7)

• Inner Product: This product also called interior product denoted by ., is used
to give the notion of orthogonality between two multivectors [4]. Let A be a
a-vector and B be a b-vector, then A.B is the subspace of B, with a (b − a)
dimension, orthogonal to subspace A. If b < a then A.B = 0 and A is orthogonal
to B. For 1-vectors, the inner product equals the scalar product used in linear
algebra V m .
Ar .Bs = Ar Bs |r−s| . (6.8)
• Scalar Product: This product denoted by ∗, is used to define distances and
modulus.
A ∗ B ≡ AB0 .

1 Inthe following, little letters will be used to represent 1-vectors whereas bolded capital letters
will stand for any multivectors.
154 P. Carré and M. Berthier

As said before, geometric algebras allows to handle geometric entities with

the formalism of algebras. In our case, we study the representation of colours by
1-vectors of R3,0 . Here is shown the translation of classical geometric transforma-
tions into the algebraic formalism R3,0 .
Let v1 , v2 , vt , v , v⊥ and vr be 1-vectors of R3,0 , the transcription into algebraic
formalism is given here for:
• Translation: vt = v1 + v2 is the result of the translation of v1 by v2 .
• Projection: v = vv2 = (v1 ∧ v2 )v−1
2 is the v1 ’s projection on v2 .
• Rejection: v⊥ = vv⊥2 = (v1 .v2 )v−12 is the v1 ’s rejection with respect to v2 .
• Reflection: vr = vvr 2 = v2 v1 v−1
2 is the v1 ’s reflection with respect to v2 .

[Link] Clifford Colour Transform

We will now introduce how geometric algebras can be associated with colour image
processing, but first of all, we give a survey on what have been done already linking
image processing and geometric algebra.
We propose to use the geometric transformations to express hue, saturation and
value colour information from any colour pixel m incoded as a 1-vector of R3,0 in
RGB colour space. This resolution is performed using only algebraic expressions in
R3,0 and is a generalization of what was already done in the quaternionic formalism.
A colour image is seen as a function f from R2 with values in the vector part of the
Clifford algebra R3,0 :

f : [x, y] −→ fr [x, y]e1 + f2 [x, y]e2 + f3 [x, y]e3 . (6.9)

Let ϑ be the 1-vector carrying the grey level axis, r carries the pur red vector and
m represents any colour vector.
• Value is the modulus of the projection of m with respect to ϑ , it is then
expressed by:
V = |(m.ϑ )ϑ −1 |. (6.10)
• Saturation is the distance from the vector m to the grey level axis ϑ , it is then the
modulus of the rejection of m with respect to ϑ :

S = |(m ∧ ϑ )ϑ −1 |. (6.11)

• To reach the hue, we need to define a colour which hue is zero, let ϑ2 be the
1-vector that represents H = 0. A general agreement is to say that pur red has
a null hue, ϑ2 is then r’s rejection with respect to ϑ . Therefore, H is the angle
between ϑ2 and mϑ⊥ = (m ∧ ϑ )ϑ −1 which is the m’s rejection with respect to ϑ .
The hue can then be given by:
 
−1 m⊥
H = cos ·ϑ .
|m⊥ |
6 Color Representation and Processes with Clifford Algebra 155

Fig. 6.4 Hue’s modification: (a) original image (b) modified image

We thus have formulated the hue, saturation and value of a RGB colour vector
using only algebraic expressions. From these concepts, we can define colour
transform by using only algebraic expressions.
Performing the translation operator along the grey axis ϑ with coefficient α ∈
R on every pixel of an image will result on an alteration of the general value or
brightness of the original image. The result of such an alteration of the brightness
seems more contrasted and warmer than the original image.
∗
m = m + αϑ = I ϑ + Seϑ T ϑ2 + αϑ → I  = I + α .

Here the rotation operator is applied on each pixels f [m, n] of the image around
the grey axis ϑ , this is shown in Fig. 6.4. The result is an alteration of the hue of the
original image.
θ θ θ θ θ θ
m = e−ϑ 2 meϑ 2 = Ie−ϑ 2 ϑ eϑ 2 + Se−ϑ 2 eϑ T ϑ2 eϑ 2
m = I ϑ + Seϑ (T+θ ) ϑ2 −→ T  = T + θ .

Figure 6.4 shows this kind of hue modification. The original image (Fig. 6.4a)
has been modified by the rotation around the greyscale axis with an angle of π /3.
So the red roof is now green, the green threshold is now blue, etc. Note that one can
imagine to choose any other colour vector for the rotation axis than the grey one to
perform an other operation than hue alteration.
The translation operator, associated with the weight β ∈ R on its saturation axis
f⊥ [x, y], is applied to perform an alteration of an image’s saturation. The saturation
axis f⊥ [x, y] is the rejection of f [x, y] with respect to ϑ .
∗ ∗
m = I ϑ + Seϑ T ϑ2 = m + β eϑ T ϑ2
∗
m = I ϑ + (S + β )eϑ T ϑ2 −→ S = S + β .
156 P. Carré and M. Berthier

Fig. 6.5 Saturation’s modification: (a) original image (b) modified image

Figure 6.5 illustrates this saturation’s alteration where the original image
(Fig. 6.4a) has been altered as described in the precedent equation to get the result
(Fig. 6.4b). The operation is equivalent to:
• Get pale or washed colours in opposition to the original image when the
saturation level is lowered as in the Fig. 6.5.
• Whereas when the saturation level is uppered, colours seem to be more vivid than
in the original image.
Note that all of these colour transformations can be done because colours are
encoded on the vector part of a R3,0 multivector. In fact, reflections, translations,
rotations and rejections are defined only for simple multivectors that is to say
information included in such multivectors is described on one and only one grade.
In this section, we introduced geometric algebras and studied several of their
properties which will help us through the next section where we will describe how
to use them to process digital images. We also showed that embedding colours into
R3,0 algebra allows to perform image alteration using algebraic formalisation only.
From these concepts, we now propose to study colour edge detection with Clifford
algebra.

[Link] Spatial Filtering with Clifford Algebra

The formalisation of colour information into R3,0 allows to define more complex
colour processings. Here, we describe how to use the geometric concepts studied
before to define algebraically spatial approaches such as colour edge detection.
As we have seen, Sangwine [8] defined a method to detect colour edges by
using colour pixels incoded in pur quaternions. The idea is to define a convolution
operation for quaternions and apply specific filters to perform the colour edge
detection. This method can be written with the geometric algebra formalism where
colours are represented by 1-vectors of R3,0 :

Sang[x, y] = (h1 ∗ I ∗ h2)[x, y], (6.12)

6 Color Representation and Processes with Clifford Algebra 157

and where h1 and h2 are a couple of filters which are used to perform a reflection
of every colour√with respect to the 1-vector associated with the greyscale axis ϑ =
(e1 + e2 + e3)/ 3 defined as:
⎡ ⎤ ⎡ ⎤
1 1 1 1 1 1
1⎣ 1⎣
h1 = 0 0 0⎦ and h2 = 0 0 0 ⎦. (6.13)
6 6 −1 −1 −1
ϑ ϑϑ ϑ ϑ ϑ

Note that these are vertical filters to detect horizontal edges.

As for the Quaternion formalism, this fulfills an average vector of every pixel in
the filters neighbourhood reflected with respect to the greyscale axis. The filtered
image is a greyscale image almost everywhere, because in homogeneous regions
the vector sum of one pixel to its neighbours reflected by the grey axis has a low
saturation in other words low distance from the greyscale axis and edges are thus
coloured due to this high distance. The limit of this method is that the filters can be
applied horizontally from left to right and from right to left for example depending
on the filters definitions but without giving the same results.
To counter this side effect involving the direction applied to the filters from
Sangwine’s method, we propose to illustrate a simple modification [9] of the
filtering scheme, with the computation of the distance between the Sangwine’s
comparison vector to the greyscale axis. This time, it gives the same results if the
filters are applied clockwise or anticlockwise. Because we are looking for a distance
between Sangwine’s comparison vector which is a colour vector to the greyscale
axis, the result is a gradient of saturation. Thus the expression of the saturation
gradient is given by:

SatGrad[x, y] = |(Sang[x, y] ∧ ϑ )ϑ −1|. (6.14)

Figure 6.6 illustrates the saturation filtering scheme. We observe the detection
of all chromatic edges. But, the main drawback of this method is still that it is
based on a saturation measurement only. In fact, when edges contain achromaticity
information only, this approach is not able to detect them properly.
The geometric product allow us to fulfill the drawback of the previous method
by describing geometrically every colour pixel f [x, y] with respect to the greyscale
axis. This description is given by this geometric product f [x, y]ϑ where f [x, y]ϑ is
broken into two terms for every pixels:

f [x, y]ϑ = f [x, y].ϑ + f [x, y] ∧ ϑ

7 89 : 7 89 :
scalar bivector

= Scal[x, y] + Biv[x, y]. (6.15)

158 P. Carré and M. Berthier

Fig. 6.6 Saturation gradient approach: (a) original image, (b) saturation gradient (c) edges
selected by maxima extraction

In order to compute the geometric product and the Sangwine filtering, we use
this couple of filters
⎛ ⎞ ⎛ ⎞
1 1 1 1 1 1
v=⎝ 0 1 0 ⎠ and u=⎝ 0 ϑ 0 ⎠.
ϑϑ ϑ ϑ ϑ ϑ −1
−1 −1

The result of the filtering is:

g[m, n] = {[ϑ f [m+1, n+1]ϑ −1 + f [m+1, n−1]]+[ϑ f [m, n + 1]ϑ −1 + f [m, n − 1]]
+[ϑ f [m − 1, n + 1]ϑ −1 + f [m − 1, n − 1]]} + f [m, n]ϑ . (6.16)

Figure 6.7 shows the AG filtering scheme.

We disjoin the scalar and bivectorial parts. The modulus of the bivectorial part
is calculated to discover where information is achromatic only (Fig. 6.7b). In fact,
the less the modulus of this bivector is, the less the saturation of the pixel is so
information is achromatic.
Next step is to apply a threshold on this modulus map to get a mask. This mask
will emphasise the areas where the modulus is low so where information is mainly
intensity in contrast to elsewhere where the colour pixels are full of chromaticity
information.
As the scalar part is the projection of every pixel on the greyscale axis μ , it thus
represents intensity information. A Prewitt filtering is applied on this scalar part
for every pixels in horizontal, vertical and diagonal directions to get an intensity or
value gradient.
P1,2,3,4 [x, y] = (G1,2,3,4 ∗ Scal[x, y]) (6.17)
6 Color Representation and Processes with Clifford Algebra 159

Fig. 6.7 (a) Original image with chromatic and achromatic information; (b) bivector part | f [x, y] ∧
ϑ |; (c) Prewitt filtering applied on the scalar part and combined with the achromatic mask; (d) final
result

with
⎡ ⎤
−1 −1 −1
G1 = ⎣ 0 0 0 ⎦
1 1 1
and G2,3,4 rotated version of G1 .
Then, to store just achromatic information, we use the mask defined with the
modulus of the bivectial part. The result gives the gradient of pixels which do not
contain chromaticity information (Fig. 6.7c). The last step is to combine this value
gradient to the saturation one defined before. We can use different techniques to
merge the two gradients as the maximum operator to preserve only the maximum
value between those two gradients (Fig. 6.7d).
160 P. Carré and M. Berthier

Fig. 6.8 Gradient examples on colour images: (a) (b) (c) original images; (d) (e) (f) final gradient

Figure 6.8 shows results on classical digital colour processing images. One can
note that the computer graphics image (Fig. 6.8a) points up that achromatic regions
are here well detected (Fig. 6.8d). The following house image (Fig. 6.8b) also
includes achromatic areas such as the gutters and the windows frame which appear
in the calculated gradient (Fig. 6.8e).
We now describe the use of these new concepts for the definition of a colour
Fourier transform.

6.3 The Colour Fourier Transform

The Fourier transform is well known to be an efficient tool for analyzing signals
and especially grey level images. When dealing with nD images, as colour images,
it is not so clear how to define a Fourier transform which is more than n Fourier
transforms computed marginally. The first attempt to define such a transform is due
to S. Sangwine and T. Ells [10] who proposed to encode the colour space RGB by
the space of imaginary quaternions H0 . For a function f from R2 to H0 representing
a colour image, the Fourier Transform is given by

Fμ f (U) = f (X) exp(− μ X,U)dX, (6.18)
R2
6 Color Representation and Processes with Clifford Algebra 161

where X = (x1 , x2 ), U = (u1 , u2 ) and μ is a unit imaginary quaternion. In this

expression μ (satisfying μ 2 = −1) is a parameter which corresponds to a privileged
direction of analysis (typically μ is the unit quaternion giving the grey axis in
[10]). It is a fact that will be discussed later that such a choice must be taken into
account.
It appears that quaternions is a special case of the more general mathematical
notion of Clifford Algebras. As we have said, these algebras, under the name of
Geometric Algebras are widely used in computer vision and robotics [11, 12]. One
of the main topics of this part of the chapter is to describe a rigorous construction of
a Fourier Transform in this context.
There are in fact many constructions of Clifford Fourier transforms with several
different motivations. In this chapter we focus on applications to colour image
processing. The Clifford definition relies strongly on the notion of group actions
which is not the case for the already existing transforms. This is precisely this
viewpoint that justifies the necessity of choosing an analyzing direction. We
illustrate how to perform frequencies filtering in colour images.

6.3.1 First Works on Fourier Transform in the Context

of Quaternion/Clifford Algebra

To our knowledge, the only generalizations of the usual Fourier transform using
quaternions and concerning image processing are those proposed by Sangwine et al.
and by Bülow. The first one is clearly motivated by colour analysis and the second
one aims at detecting two-dimensional symmetries in grey-level images.
Several constructions have been proposed in the context of Clifford algebras.
In [13], a definition is given using the algebras R2,0 and R3,0 in order to introduce
the concept of 2D analytic signal. A definition appears also in [14] which is
mainly applied to analyse frequencies of vector fields. With the same Fourier kernel
Mawardi and Hitzer in [15] establish in [15] an uncertainty principle for multivector
functions. The reader may find in [16] a construction using the Dirac operator
and applications to Gabor filters. Let us also mention, from a different viewpoint,
reference [17] where generalized Fourier descriptors are defined by considering the
action of the motion group of R2 that is the semidirect product of the groups R2 and
SO(2).
The final part of this section describes the generalization of the Fourier Transform
by using Bivectors of the Clifford Algebra R3,0 . We start by illustrating how
Quaternion are used to define colour Fourier Transform.
162 P. Carré and M. Berthier

[Link] Quaternionic Fourier Transforms: Definition

As already mentioned the idea of [10] is to encode colour information through

imaginary quaternions. A colour image is then considered as a map f : R2 −→ H0
defined by
f (x, y) = r(x, y)i + g(x, y) j + b(x, y)k,
where r(x, y), g(x, y), and b(x, y) are the red, green and blue components of the
image. R To define a quaternionic Fourier transform the authors of [10] propose to
replace the imaginary complex i by some imaginary unit quaternion μ . It√ is easily
checked that μ 2 = −1 and a typical choice for μ is μ = (i + j + k)/ 3 which
corresponds in RBG to the grey axis. The transform is given by formula:

Fμ f (U) = f (X) exp(− μ X,U)dX. (6.19)
R2

The quaternionic Fourier coefficients are decomposed with respect to a symplec-

tic decomposition associated to μ , each one of the factors being expressed in the
polar form:
Fμ f = A exp[μθ ] + A⊥ exp[μθ⊥ ]ν
with ν an imaginary unit quaternion orthogonal to μ . The authors propose a spectral
interpretation from this decomposition (see [10] for details).
Bülow’s approach is quite different since it concerns mainly the analysis of
symmetries of a signal f from R2 to R given for example by a grey-level image.
For such a signal, the quaternionic Fourier transform proposed by Bülow reads:

Fij f (U) = exp(−2π ix1u1 ) f (X) exp(−2π ju2 x2 )dX. (6.20)
R2

Note that i and j can be replaced by arbitrary pur imaginary quaternions. The choice
of this formula is justified by the following equality:

Fij f (U) = Fcc f (U) − iFsc f (U) − jFcs f (U) + kFss f (U),

where

Fcc f (U) = f (X) cos(2π u1 x1 ) cos(2π u2x2 )dX
R2

and similar expressions involving sinus and cosinus for Fsc f , Fcs f and Fss f .
We refer the reader to [18] for details and applications to analytic signals.

[Link] Quaternionic Fourier Transforms: Numerical Analysis

In order to understand what the Fourier coefficients stand for, we studied the digital
caracterization of the Discrete Quaternionic Fourier Transform (DQFT). The colour
6 Color Representation and Processes with Clifford Algebra 163

Fourier spectrum presented some symmetries due to zero scalar spatial part of any
colour image exactly as it was well known that the spectrum of a real signal by a
complex Fourier transform (CFT) had hermitian properties of symmetry. Even if
the spatial information of a colour image is using pure quaternions only, applying
a DQFT on an image results in full quaternions (i.e., with scalar part non zero).
We wanted to find, after Inverse DQFT, a space where scalar part is zero in order
to avoid any loss of information as the spatial colour image is coded on a pure
quaternion matrix which real part automatically set to zero.
Let
F[o, p] = Fr [o, p] + Fi[o, p]i + Fj [o, p] j + Fk [o, p]k (6.21)

be the spectral quaternion at coordinates (o, p) ∈ ([ −N −M M

2 + 1.. 2 ], [ 2 .. 2 ]) and
N

1
∑ ∑ e2μπ ( M + N ) F[o, p]
om pn
f [m, n] = √ (6.22)
MN o p

the Inverse DQFT quaternion of (m, n) spatial coordinates.

Developing this, with μ = μi i + μ j j + μk k, the cartesian real part form of the spatial
domain leads to
1    om pn 
fr [m, n] = √ ∑ ∑ cos 2π M + N Fr [o, p]
MN o p
  om pn  
− sin 2π + (μi Fi [o, p] + μ j Fj [o, p] + μk Fk [o, p]) (6.23)
M N
fr (s,t) is null when Fr [0, 0] = Fr [ M2 , N2 ] = 0 and for all o ∈ [1; M2 − 1] and p ∈
[1; N2 − 1]:
Fr [−o, −p] = −Fr [o, p] (6.24)

Moreover for all o ∈ [0; M2 ] and p ∈ [0; N2 ]:

Fi [−o, −p] = Fi [o, p]

Fj [−o, −p] = Fj [o, p]
Fk [−o, −p] = Fk [o, p]. (6.25)

We can see that the real part must be odd and all the imaginary parts must be
even. This is a direct extension of the antihermitian property of the complex Fourier
transform of imaginary signal.
When studying the complex spectrum domain, several notions are helpful such
as the modulus and the angle.
A Fourier coefficient, F[o, p] = q0 + iq1 + jq2 + kq3 can be written as:

F[o, p] = q = |q|eνϕ
164 P. Carré and M. Berthier

Fig. 6.9 Polar representation of the quaternionic Fourier transform

with |q| the QFT modulus, ϕ ∈ R the QFT phase and ν ∈ H0 ∩ H1 the QFT axis.
Figure 6.9 illustrates this polar representation of the Fourier coefficient for the image
Lena. We can see that the Fourier coefficient has a modulus similar to that Greyscale
image. It is more difficult to give interpretation of the information contained in the
angle or the axis.
In order to try to give an interpretation of the information contained in the
quaternionic spectrum of colour images, we can study spatial atoms associated with
a pulse (Dirac) in the frequency domain.
Initialization could be done in two different ways:
• F[o, p] = Kr .δo0 ,p0 [o, p] − Kr δ−o0 ,−p0 [o, p]. Initialization done on the real part
of the spectrum, leading to odd oscillations on the spatial domain linked to the
direction μ parameter of the Fourier transform. Complex colours are obtained
in the RGB colour space after modifying this μ parameter and normalising it
because it always needs to stay a pure unit quaternion.
Fr [o0 , p0 ] = Kr and Fr [−o0, −p0 ] = −Kr is associated with:
  o m p n 
0 0
f [m, n] = 2 μ (Kr ) sin 2π + . (6.26)
M N
Initializing a pair of constants on the real component leads to a spatial oscillation
following the same imaginary component(s) as those included in the direction μ
(Fig. 6.10b).
• F[o, p] = e.(Ke .δo0 ,p0 [o, p] + Ke δ−o0 ,−p0 [o, p]) with e = i, j or k. Initialization
done on the imaginary part of the spectrum, leading to even oscillations on the
spatial domain independently from the μ parameter of the Fourier transform.
Complex colours in the RGB colour space are reached by initialization on several
imaginary components weighted as in the additive colour synthesis theory.
With e = i, j, k, Fe [o0 , p0 ] = Fe [−o0 , −p0 ] = Ke is associated with:
   o m p n 
0 0
f [m, n] = e 2(Ke ) cos 2π + . (6.27)
M N
Initializing a pair of constants on any imaginary component with any direction μ
leads to a spatial oscillation on the same component (Fig. 6.10a).
6 Color Representation and Processes with Clifford Algebra 165

Fig. 6.10 Spectrum initialization examples

The coordinates (o0 , p0 ) and (−o0 , −p0 ) of the two initialization points in the
Fourier domain affect the orientation and the frequency of the oscillations in the
spatial domain as it does so with greyscale image in complex Fourier domain.
Orientation of the oscillations can be changed as shown in Fig. 6.10c.
Below we outline the different ways of defining a Clifford Fourier Transform.

[Link] Clifford Fourier Transforms

In [13], the Clifford Fourier transform is defined by


Fe1 e2 f (U) = exp(−2π e1e2 U, X) f (X)dX (6.28)
R2

for a function f (X) = f (xe1 ) = f (x)e2 from R to R and


Fe1 e2 e3 f (U) = exp(−2π e1 e2 e3 U, X) f (X)dX (6.29)
R3

for a function f (X) = f (x1 e1 + x2 e2 ) = f (x1 , x2 )e3 from R2 to R. The coefficient

e1 e2 , resp. e1 e2 e3 , is the so-called pseudoscalar of the Clifford algebra R2,0 ,
resp. R3,0 . These transforms appear naturally when dealing with the analytic and
monogenic signals.
Scheuermann et al. also use the last kernel for a function f : R3 −→ R3,0 :

Fe1 e2 e3 f (U) = f (X) exp(−2π e1e2 e3 U, X)dX. (6.30)
R3

Note that if we set

f = f 0 + f 1 e1 + f 2 e2 + f 3 e3 +
+ f23 i3 e1 + f31 i3 e2 + f12 i3 e3 + f123 i3
166 P. Carré and M. Berthier

with i3 = e1 e2 e3 , this transform can be written as a sum of four complex Fourier

transforms by identifying i3 with the imaginary complex i. In particular, for a
function f with values in the vector part of the Clifford algebra, this reduces to
marginal processing. This Clifford Fourier transform is used to analyse frequencies
of vector fields and the behavior of vector valued filters.
The definition proposed in [16] relies on Clifford analysis and involves the so-
called angular Dirac operator Γ. The general formula is
 n   π 
1
F± f (U) = √ exp ∓i ΓU × exp(−i < U, X >) f (X)dX. (6.31)
2π Rn 2

For the special case of a function f from R2 to C2 = R0,2 ⊗ C, the kernel can be
made explicit and

1
F± f (U) = exp(±U ∧ X) f (X)dX. (6.32)
2π R2

Let us remark that exp(±U ∧ X) is the exponential of a bivector, i.e., a spinor. This
construction allows to introduce two-dimensional Clifford Gabor filters (see [16] for
details).
As the reader may notice, there are many ways to generalize the usual definition
of the complex Fourier transform. In all the situations mentioned above the
multiplication is non commutative and as a consequence the position of the kernel
in the integral is arbitrary. We may in fact distinguish two kinds of approaches: the
first ones deal with so called bivectors (see below) and the second ones involve the
pseudoscalar e1 e2 e3 of the Clifford algebra R3,0 . The rest of this paper focus on
the first approaches. The purpose of the last part of this chapter is to propose a well
founded mathematical definition that explains why it is necessary to introduce those
bivectors and their role in the definition. Before going into details, we recall some
mathematical notions.

6.3.2 Mathematical Background

[Link] Mathematical Viewpoint on Fourier Transforms

We start by some considerations about the theory of abstract Fourier transform and
then introduce basic notions on Clifford algebras and spinor groups. The main result
of this section is the description of the Spin(3) and Spin(4) characters. From the
mathematical viewpoint, defining a Fourier Transform requires to deal with group
actions. For example, in the classical one-dimensional formula
 +∞
F f (u) = f (x) exp(−iux)dx (6.33)
−∞
6 Color Representation and Processes with Clifford Algebra 167

the involved group is the additive group (R, +). This is closely related to the
well-known Shift Theorem

F fα (u) = F f (u) exp(iuα ) (6.34)

where fα (x) denotes the function x −→ f (x + α ), which reflects the fact that a
translation of a vector α produces a multiplication by exp(iuα ). The correspondance
α −→ exp(iuα ) is a so-called character of the additive group (R, +).
More precisely, a character of an abelian group G is a map ϕ : G −→ S1 that
preserves the composition laws of both groups. Here S1 is the multiplicative group
of unit complex numbers. It is a special case, for abelian groups, of the notion
of irreducible unitary representations, [19]. The abstract definition of a Fourier
Transform for an (abelian) additive group G and a function f from G to C is given by

F f (ϕ ) = f (x)ϕ (−x)dν (x), (6.35)
G

where ϕ is a character and ν is a measure on G.

The characters of the group (Rn , +) are the maps

X = (x1 , . . . , xn ) −→ exp(u1 x1 + · · · + unxn )

parametrized by U = (u1 , . . . , un ). They form the group (Rn , +). Applying the above
formula to this situation leads to the usual Fourier Transform

F f (U) = f (X) exp(−iU, X)dX. (6.36)
Rn

It is classical, see [19], that considering the group of rotations SO(2, R), resp. the
group Zn , and the corresponding characters yields to the Fourier series theory, resp.
the discrete Fourier Transform.
One of the ingredients of the construction of the Colour Fourier transform is the
notion of Spin characters which extends the notion of characters to maps from R2
to spinor groups representing rotations.

[Link] Rotation

In the same way that characters of the Fourier transform for grey-level images are
maps form R2 to the rotation group S1 of the complex plane C, we want to define
characters for colour images as maps from R2 to the rotation group acting on the
space of colours, chosen in the sequel to be RGB. The Clifford algebra framework
is particulary well adapted to treat this problem since it allows to encode geometric
transformations via algebraic formulas.
Rotations of R3 correspond to specific elements of R3,0 , namely those given by

τ = a1 + be1e2 + ce2 e3 + de1 e3

168 P. Carré and M. Berthier

with a2 + b2 + c2 + d 2 = 1. These are called spinors and form a group Spin(3)

isomorphic to the group of unit quaternions. The image of a vector v under a rotation
given by some spinor τ is the vector

τ ⊥v := τ −1 vτ . (6.37)

As it is more convenient to define the usual Fourier transform in the complex

setting, it is more convenient for the following to consider the colour space as
embedded in R4 . This simplifies, in particular, the implementation through a double
complex FFT. The Clifford algebra R4,0 is the vector space of dimension 16 with
basis given by the set

{ei1 · · · eik , i1 < · · · < ik ∈ [1, . . . , 4]}

and the unit 1 (the vectors e1 , e2 , e3 and e4 are elements of an orthonormal basis
of R4 ). As before, the multiplication rules are given by e2i = 1 and ei e j = −e j ei .
The corresponding spinor group Spin(4) is the cross product of two copies of
Spin(3) and acts as rotations on vectors of R4 by formula (6.37).
One fundamental remark is that every spinor τ of Spin(3), resp. Spin(4), can be
written as the exponential of a bivector of R3,0 , resp. R4,0 , i.e.,

1 i
τ=∑ B (6.38)
i≥0 i!

for some bivector B. This means precisely that the Lie exponential map is onto (see
[20] for a general theorem on compact connected Lie groups). As an example, the
spinor
 
(1 + n2n1 ) n2 ∧ n1
τ=! = exp (θ /2)
2(1 + n1 · n2 ) |n2 ∧ n1 |

is the rotation of R3 that sends by formula (6.37) the unit vector n1 to the unit vector
n2 leaving the plane (n1 , n2 ) globally invariant. In the above expression, θ is the
angle between n1 and n2 and |n2 ∧ n1 | is the magnitude of the bivector n2 ∧ n1 .

[Link] Spin Characters

The aim here is to compute the group morphisms (i.e., maps preserving the
composition laws) from the additive group R2 to the spinor group of the involved
Clifford algebra. We don’t detail the proofs since they need specific tools on Lie
algebras (see [21] for explanations). In the sequel, we denote S23,0 , resp. S24,0 , the set
of unit bivectors of the algebra R3,0 , resp. R4,0 . Let us first treat the case of Spin(3)
characters.
6 Color Representation and Processes with Clifford Algebra 169

Theorem 1 (Spin (3) Characters). The group morphisms of the additive group R2
to Spin(3) are given by the maps that send (x1 , x2 ) to:
 
1
exp (x1 u1 + x2 u2 )B , (6.39)
2

where B belongs to S23,0 and u1 and u2 are reals.

It is important to notice that if the Spin(3) characters are parametrized as usual
by two frequencies u1 and u2 , they are also parametrized by a unit bivector B of the
Clifford algebra R3,0 . We have already mentionned that, for implementation reasons,
it is preferable to deal with the Clifford algebra R4,0 and the corresponding Spin(4)
group. Using the fact that this one is the cross product of two copies of Spin(3), one
can prove the following result.
Theorem 2 (Spin (4) Characters). The group morphisms of the additive group R2
to Spin(4) are given by the maps that send (x1 , x2 ) to:
   
1 1
exp [x1 (u1 + u3 ) + x2(u2 + u4)]D exp [x1 (u1 − u3) + x2 (u2 − u4)]I4 D ,
2 2
(6.40)
where D belongs to S24,0 and u1 , u2 , u3 , and u4 are reals. In this expression I4 denotes
the pseudo scalar e1 e2 e3 e4 of the algebra R4,0 .
Let us make few comments. The first one is that Spin(4) characters are
parametrized by four frequencies and a bivector of S24,0 . This is not really suprising
in view of the classification of rotation in R4 (see below). The second one concerns
the product I4 D of the pseudo scalar I4 by the bivector D. A simple bivector D
(i.e., the exterior product of two vectors) represents a piece of a two-dimensional
vector space of R4 with a magnitude and an orientation. Multiplying this one by I4
consists in fact to consider the element of S24,0 which represents the piece of vector
space orthogonal to D in R4 (see [22]). The spinor is written as a product of two
commuting spinors each one acting as a rotation (the first one in the D plane, the
second one in the I4 D plane). Finally, note that these formulas are quite natural and
generalize the usual formula since the imaginary complex i can be viewed as the
unit bivector coding the complex plane.
We denote ϕ(u1 ,u2 ,u3 ,u4 ,D) the morphisms given by equation (6.40).

[Link] About Rotations of R4

The reader may find in [21] the complete description of the rotations in the space
R4 . The classification is given as follows.
• Simple rotations are exponential of simple bivectors that is exterior products of
two vectors. These rotations turn only one plane.
170 P. Carré and M. Berthier

• Isoclinic rotations are exponential of simple bivectors multiplied by one of the

elements (1 ± I4)/2. An isoclinic rotation has an infinity of rotation planes.
• General rotations have two invariant planes which are completely orthogonal
with different angle of rotation.
In the Clifford algebra R3,0 every bivector is simple, i.e., it represents a piece of
plane. Formula (6.39) describes simple rotations. Formula (6.40) describes general
rotations in R4 . In the next section, we make use of the following special Spin(4)
characters:
(x1 , x2 ) −→ ϕ(u1 ,u2 ,0,0,D) (x1 , x2 ). (6.41)
They correspond to isoclinic rotations.

6.3.3 Clifford Colour Fourier Transform with Spin Characters

Before examining Clifford Colour Fourier, it is useful to rewrite the usual definition
of the complex Fourier transform in the language of Clifford algebras.

[Link] The Usual Transform in the Clifford Framework

Let us consider the usual Fourier formula when n equals 2:


F f (u1 , u2 ) = f (x1 , x2 ) exp(−i(x1 u1 + x2u2 ))dx1 dx2 . (6.42)
R2

The involved characters are the maps

(x1 , x2 ) −→ exp(i(x1 u1 + x2u2 ))

with values in the group of unit complex numbers which is in fact the group Spin(2)
of the Clifford algebra R2,0 . Considering the complex valued function f = f1 + i f2
as a map in the vector part of this algebra, i.e.,

f (x1 , x2 ) = f1 (x1 , x2 )e1 + f2 (x1 , x2 )e2

the Fourier transform may be written


F f (u1 , u2 ) = {[cos((x1 u1 + x2 u2 )/2) + sin((x1 u1 + x2u2 )/2)e1 e2 ]
R2
[ f1 (x1 , x2 )e1 + f2 (x1 , x2 )e2 ] [cos(−(x1 u1 + x2u2 )/2)
+ sin(−(x1 u1 + x2 u2 )/2)e1 e2 ]} dx1 dx2 . (6.43)
6 Color Representation and Processes with Clifford Algebra 171

If we consider the action ⊥ introduced in formula (6.37), we obtain:


F f (u1 , u2 ) = [ f1 (x1 , x2 )e1 + f2 (x1 , x2 )e2 ] ⊥ϕ(u1 ,u2 ,e1 e2 ) (−x1 , −x2 )dx1 dx2 ,
R2
(6.44)
where  
1
ϕ(u1 ,u2 ,e1 e2 ) (x1 , x2 ) = exp (x1 u1 + x2 u2 )e1 e2
2
since as said before, the imaginary complex i corresponds to the bivector e1 e2 . We
now describe the generalization of this definition.

[Link] Definition of the Clifford Fourier Transform

We give first a general definition for a function f from R2 with values in the vector
part of the Clifford algebra R4,0 :

f : (x1 , x2 ) −→ f1 (x1 , x2 )e1 + f2 (x1 , x2 )e2 + f3 (x1 , x2 )e3 + f4 (x1 , x2 )e4 . (6.45)

Definition 1 (General Definition). The Clifford Fourier transform of the function

f defined by (6.45) is given by

CF f (u1 , u2 , u3 , u4 , D) = f (x1 , x2 )⊥ϕ(u1 ,u2 ,u3 ,u4 ,D) (−x1 , −x2 )dx1 dx2 . (6.46)
R2

It is defined on R4 × S24,0 .
Let us give an example. The vector space H of quaternions can be identified with
the vector space R4 under the correspondance: e1 ↔ i, e2 ↔ j, e3 ↔ k, and e4 ↔ 1.
It then can be shown that

Fi j f (u1 , u2 ) = CF f (2π u1 , 0, 0, 2π u2, Di j ),

where Fi j is the quaternioninc transform of Bülow and Di j is the bivector

1
Di j = − (e1 + e2 )(e3 − e4 ).
4
For most of the applications to colour image processing that will be investigated
below, it is sufficient to consider a transform that can be applied to functions with
values in the vector part of the algebra R3,0 . Such a function is given by

f : (x1 , x2 ) −→ f1 (x1 , x2 )e1 + f2 (x1 , x2 )e2 + f3 (x1 , x2 )e3 + 0e4 (6.47)

just as a real function is a complex function with 0 imaginary part.

172 P. Carré and M. Berthier

Definition 2 (Definition for Colour Images). The Clifford Fourier transform of

the function f defined by (6.47) in the direction D is given by

CF D f (u1 , u2 ) = f (x1 , x2 )⊥ϕ(u1 ,u2 ,0,0,D) (−x1 , −x2 )dx1 dx2 . (6.48)
R2

It is defined on R2 .
As an example, let us mention that (under the above identification of H with R4 )

Fμ f (u1 , u2 ) = CF Dμ f (u1 , u2 ),

where Fμ is the quaternionic transform of Sangwine et al. and Dμ is the bivector

D μ = ( μ1 e1 + μ2 e2 + μ3 e3 ) ∧ e4

with μ = μ1 i + μ2 j + μ3k a unit imaginary quaternion.

Both definitions involve bivectors of S24,0 (as variable and as parameter). We give
now some of the properties satisfied by the Clifford Fourier transform.

[Link] Properties of the Clifford Fourier Transform

A parallel and orthogonal decomposition, very closed to the symplectic decom-

position used by Sangwine, is used to study the properties of the colour Fourier
Transform.

Parallel and Orthogonal Decomposition

The function f given by equation (6.47) can be decomposed as

f = f D + f ⊥D , (6.49)

where fD , resp. f⊥D , is the parallel part, resp. the orthogonal part, of f with respect
to the bivector D. Simple computations show that

CF f (u1 , u2 , u3 , u4 , D)

= fD (x1 , x2 ) exp [−(x1 (u1 + u3 ) + x2(u2 + u4))D] dx1 dx2
R2

+ f⊥D (x1 , x2 ) exp [−(x1 (u1 + u3 ) + x2(u2 + u4 ))I4 D] dx1 dx2 . (6.50)
R2

Applying this decomposition to colour images, leads to the following result.

6 Color Representation and Processes with Clifford Algebra 173

Proposition 1 (Clifford Fourier Transform Decomposition for Colour Images).

Let f be as in (6.47), then

CF D f = CF D ( fD ) + CF D ( f⊥D ) = (CF D f )D + (CF D f )⊥D . (6.51)

In practice, the decomposition is obtained as follows. Let us fix a simple bivector

D = v1 ∧ v2 of S24,0 . There exists a vector w2 of R4 , namely w2 = v−1 1 (v1 ∧ v2 ) =
v−1
1 D, such that
D = v1 ∧ v2 = v1 ∧ w2 = v1 w2 .
In the same way, if v3 is a unit vector such that v3 ∧ I4 D = 0, then the vector w4 =
v−1
3 I4 D satisfies
I4 D = v3 ∧ w4 = v3 w4 .
This precisely means that if v1 and v3 are chosen to be unit vectors (in this case,
v−1 −1
1 = v1 and v3 = v3 ), the set (v1 , w2 , v3 , w4 ) is an orthonormal basis of R adapted
4

to D and I4 D. We can then write a function f

f (x1 , x2 ) = [( f (x1 , x2 ) · v1 )v1 + ( f (x1 , x2 ) · (v1 D))v1 D]

+ [( f (x1 , x2 ) · v3 )v3 + ( f (x1 , x2 ) · (v3 I4 D))v3 I4 D] (6.52)

or equivalently
f (x1 , x2 ) = v1 [( f (x1 , x2 ) · v1 ) + ( f (x1 , x2 ) · (v1 D))D]
+ v3 [( f (x1 , x2 ) · v3 ) + ( f (x1 , x2 ) · (v3 I4 D))I4 D]
= v1 [α (x1 , x2 ) + β (x1, x2 )D] + v3 [γ (x1 , x2 ) + δ (x1 , x2 )I4 D] (6.53)
Since D2 = (I4 D)2 = −1, the terms in the brackets can be identified with complex
numbers α (x1 , x2 ) + iβ (x1 , x2 ) and γ (x1 , x2 ) + iδ (x1 , x2 ) on which a usual complex
FFT can be applied. Let us denote α ; (u1 , u2 ) + iβ;(u1 , u2 ) and γ;(u1 , u2 ) + iδ;(u1 , u2 )
the results. The Clifford Fourier transform of f in the direction D is given by
   
CF D f (u1 , u2 ) = v1 α; (u1 , u2 ) + β;(u1 , u2 )D + v3 γ;(u1 , u2 ) + δ;(u1 , u2 )I4 D .
(6.54)
For the applications treated below, it will be clear how to choose the unit vectors
v1 and v3 .

Inverse Clifford Fourier Transform

The Clifford Fourier transform defined by equation (6.46) is left invertible. Its
inverse is given by
CF −1 g(x1 , x2 )

= g(u1 , u2 , u3 , u4 , D)⊥ϕ(u1 ,u2 ,u3 ,u4 ,D) (x1 , x2 )du1 du2 du3du4 dν (D), (6.55)
R4 ×S24,0
174 P. Carré and M. Berthier

where ν is a measure on the set S24,0 . The inversion formula for the Clifford Fourier
transform (6.48) (colour image definition) is much more simpler.
Proposition 2 (Inverse Clifford Fourier Transform for Colour Images). The
Clifford Fourier transform defined by equation (6.48) is invertible. Its inverse is
given by

CF −1
D g(x1 , x2 ) = g(u1 , u2 )⊥ϕ(u1 ,u2 ,0,0,D) (x1 , x2 )du1 du2 . (6.56)
R2

Remark the analogy (change of signs in the spin characters) with the usual
inversion formula.
Since this chapter is mainly devoted to colour image processing and for sake of
simplicity, we describe now properties concerning the only transformation (6.48).

Shift Theorem

It is important here to notice that the transform CF D satisfies a natural Shift

Theorem which results in fact from the way it has been constructed. Let us denote
f(α1 ,α2 ) the function defined by

f(α1 ,α2 ) (x1 , x2 ) = f (x1 + α1 , x2 + α2 ), (6.57)

where f is as in (6.47).
Proposition 3 (Shift Theorem for Colour Images). The Clifford Fourier Trans-
form of the function f(α1 ,α2 ) in the direction D is given by

CF D f(α1 ,α2 ) (u1 , u2 ) = CF D f (u1 , u2 )⊥ϕ(u1 ,u2 ,0,0,D) (α1 , α2 ). (6.58)

Generalized Hermitian Symmetry

It is well known that a function f defined on R2 is real if and only if the usual
Fourier coefficients satisfy

F f (−u1 , −u2 ) = F f (u1 , v2 ), (6.59)

where F is the usual Fourier transform and the overline denotes the complex
conjugacy. This property, called hermitian symmetry is important when dealing with
frequencies filtering. Note that the precedent equation implies that

ℑ [F f (u1 , u2 ) exp(i(x1 u1 + x2u2 )) + F f (−u1 , −u2 ) exp(−i(x1 u1 + x2 u2 ))] = 0,

(6.60)
where ℑ denotes the imaginary part and thus that the function f is real.
6 Color Representation and Processes with Clifford Algebra 175

With the quaternionic Fourier transform, we noted that the colour Fourier
coefficients satisfied an anti-hermitian symmetry.
The next proposition generalizes this hermitian property to the Clifford Fourier
transform for color images.
Proposition 4 (Generalized Hermitian Symmetry for Colour Images). Let f be
given as in (6.47), then the e4 term in

CF D f (u1 , u2 )⊥ϕ(u1 ,u2 ,0,0,D) (x1 , x2 ) + CF D f (−u1 , −u2 )⊥ϕ(−u1 ,−u2 ,0,0,D) (x1 , x2 )
(6.61)

is zero. Moreover, the expression does not depend on D.

This proposition justifies the fact that the masks used for filtering in the frequency
domain are chosen to be invariant with respect to the transformation (u1 , u2 ) −→
(−u1 , −u2 ).

Energy Conservation

The following statement is an analog of the usual Parceval equality satisfied by the
usual Fourier transform.
Proposition 5 (Clifford Parceval Equality). Let f be given as in (6.47), then
 
(CF D f (u1 , u2 ))2 du1 du2 = ( f (x1 , x2 ))2 dx1 dx2 (6.62)
R2 R2

whenever one term is defined (and thus both terms are defined).
Let us recall that for a vector u of the algebra R4,0 , u2 = Q(u) where Q is the
Euclidean quadratic form on R4 .

[Link] Examples of the Use of the Colour Spectrum by Frequency

Windowing

The definition of the Colour Fourier transform involves explicitly a bivector D of

R4,0 which, as already said, corresponds to an analyzing direction. We precise now
what kinds of bivectors may be considered. We only deal here with simple bivectors,
i.e., bivectors that are wedge products of two vectors of R4 . These one correspond
to pieces of two-dimensional subspaces of R4 with a magnitude and an orientation.
In the Fourier definition, the bivector D is of magnitude 1.
176 P. Carré and M. Berthier

Colour Bivector

Let μ = μ1 e1 + μ2 e2 + μ3 e3 be a unit colour of the cube RGB. The bivector

corresponding to this colour is given by
D μ = μ ∧ e4 . (6.63)
The parallel part of the Clifford Fourier transform CF Dμ can be used to analyse the
frequencies of the colour image in the direction μ , whereas the orthogonal part can
be used to analyse frequencies of colours that are orthogonal to μ (see examples
below).

Hue Bivector

Proposition 6. Let H be the set of bivectors

H = {(e1 + e2 + e3 ) ∧ μ , μ ∈ RGB} (6.64)

with the quivalence relation

D1  D2 ⇐⇒ D1 = λ D2 f or some λ > 0. (6.65)

Then, H/  is in bijection with the set of hues.

It appears that choosing a unit bivector Dμ4 of the form (e1 + e2 + e3 ) ∧ μ makes
it possible to analyse the frequencies of a colour image (through the parallel part of
the Clifford Fourier transform) with respect to the hue of the colour μ .
Note that it is also possible to choose a unit bivector which is the wedge product
of two colours μ1 and μ2 .
Figure 6.11 shows the result of a directional filtering applied on a colour version
of the classical Fourier house. The original image is on the left. The bivector used on
this example is the one coding the red colour, i.e., D = e1 ∧ e4 . The mask is defined
in the Fourier domain by 0 on the set {| arg(z) − π /2| < ε } ∪ {| arg(z) + π /2| < ε }
and by 1 elsewhere. It can be seen on the right image that the horizontal red lines
have disappeared, whereas the green horizontal lines remain unchanged.
Figure 6.12 gives an illustration of the influence of the choice of the bivector
D which, as said before, corresponds to an analizing direction. The filter used in
this case is a low-pass filter in the parallel part. In the left image, D is once again
the √bivector e1 ∧ e4 coding the red color. For the right image, D is the bivector
(1/ 2)(e2 + e3 ) ∧ e1 coding the red hue.
On the left, both green and cyan stripes are not modified. This comes from the
fact that these colours belong to the orthogonal part given by I4 D. The result is
different on the right image. Cyan stripes are blurred since the bivectors representing
the red and cyan hues are opposite and thus generate the same plane. Green stripes
are no more√invariant since the vector e2 of the green colour is no longer orthogonal
to D = (1/ 2)(e2 + e3 ) ∧ e1 .
6 Color Representation and Processes with Clifford Algebra 177

Fig. 6.11 Original image—directional filtering in the red color

Fig. 6.12 Colour filtering–Hue filtering

Let us emphasize one fundamental property of the Clifford Fourier transform

defined with Spin characters. The preceeding definition can be extended by consid-
ering any positive definite quadratic form on R4 . We do not enter into details here
and just give an illustration in Fig. 6.13.
The right image is the original image on which is applied a low-pass filter in the
part orthogonal to the bivector D = e1 ∧ e4 . The middle image corresponds to the
usual Euclidean quadratic form while the image on the√right involves the quadratic
form given by the identity matrix in the basis (e1 , (1/ 2)(e1 + e2 ), iα /iα , e4 ) of
R4 , iα being the vector coding the colour α of the background leaves. This precisely
means that, in this case, the red, yellow colours, and α are considered as orthogonal.
Note that these ones √ are the dominant colours of the original image. The bivector
I4 D is given by (1/ 2)(e1 + e2 ) ∧ (iα /iα ). It contains the yellow colour and α .
178 P. Carré and M. Berthier

Fig. 6.13 Original image—Eclidean metric–adapted metric

In the middle image, the green and blue high frequencies are removed while
the red ones are preserved (I4 D = e2 ∧ e3 ). The low-pass filter removes all high
frequencies of the right image excepted those of the red petals.

6.4 Conclusion

Hypercomplex or quaternions numbers have been used recently for both greyscale
and colour image processing. Geometric algebra allows to handle geometric entities
such as scalars, vectors, or bivectors independently. These entities are handled with
the help algebraic expressions such as products (inner, outer, geometric, . . . ) for
instance and rules over these products allow to affect or modify entities.
This chapter presents how quaternion and geometric algebra is used as a new
formalism to perform colour image processing.
The first section reminds us how to use quaternions to process colour infor-
mation, and how the three components of a colour pixel split to the vectorial
part of R3,0 multivector. This condition is required to apply and define geometric
operations algebraically on colour vectors such as translations and rotations for
instance. After that, we illustrate that the R3,0 algebra is convenient to analyse and/or
alter geometrically colour in images with operations tools defined algebraically. For
that, we gave examples with alteration of the global hue, saturation, or value of
colour images. After this description of some basic issues of colour manipulations,
we shows different existent filtering approaches for using quaternions with colour
images, and we proposed to generalize approaches already defined with quaternions
and enhanced them with this new formalism. Illustrations proved it gave more
accurate colour edge detection.
The second section introduces the discrete quaternionic Fourier transform pro-
posed by Sangwine and by Bülow, and the conditions on the quaternionic spectrum
to enable manipulations into this frequency domain without loosing information
when going back to the spatial domain. This parts gives some interpretation of
the quaternionic Fourier space. We conclude on a geometric approach using group
6 Color Representation and Processes with Clifford Algebra 179

actions for the Clifford colour fourier transform. The idea is to generalize the
usual definition based on the characters of abelian groups by considering group
morphisms from R2 to spinor groups Spin(3) and Spin(4). The transformation is
parameterized by a bivector and a quadratic form, the choice of which is related to
the application to be treated.

References

1. Sangwine SJ (1996) Fourier transforms of colour images using quaternion, or hypercomplex,

numbers. Electron Lett 32(21):1979–1980
2. Sangwine SJ (1998) Colour image edge detector based on quaternion convolution. Electron
Let