1. CIECAM02 Color Appearance Model :
Color Processing as a Color Management Problem
Shereef Shehata
November 3, 2015
2. November 3, 2015 Slide 2
Agenda
Challenges
TVP9010 Display Processor
Color Spaces
CIECAM02 and Inverse CIECAM02
Color Processing
Conclusion and Future Work
4. November 3, 2015 Slide 4
Video Display Processor
De-interlacer Scaler
Detail/Edge
Enhancement
CIECAM02
Color
Processing
Inverse
CIECAM02
Source
Display
5. November 3, 2015 Slide 5
Color management is a science about color matching and color
reproduction among different devices and viewing conditions.
Examples:
Color matching between a monitor and a printer
Color matching between a camera and a monitor
Color matching between D65 light and fluorescent light
The color management applies to whole color reproduction
chain: capture, transmission and display.
Two goals: Color accuracy and pleasing picture
What is Color Management?
6. November 3, 2015 Slide 6
CIECAM02 Color Appearance Model
CIECAM02 is the state of art color appearance model rectified
by CIE in 2002. It combines the best attributes and ideas of
diverse color models into a unified color model.
CIECAM02 is perceptually uniform color model, aiming to
model how people perceive colors.
CIECAM02 has linear constant hue lines and concentric
saturation lines.
CIECAM02 also comprehends how viewing and display
conditions affect the perception of colors.
The computation of CIECAM02 and its inverse is moderate.
7. November 3, 2015 Slide 7
Color Spaces
YCbCr
Coding , transmission and storage
RGB
Two types: Standard (NTSC, 709) and device-dependent (DLP,
LCD panels, CRT)
XYZ
Tristimulus. A special RGB color space based on spectrally pure
primaries and CIE color matching procedure. Device-independent.
Standard color space to define other RGB color spaces
LMS
Cone responses
Perceptual color spaces
LAB
LUV
CIECAM97
CIECAM02
HSV is not a perceptual color space
8. November 3, 2015 Slide 8
RGB Color Spaces
HDTV (709), Modern NTSC
1953 NTSC (Obsolete)
⋅
=
709
709
709
950227.0119193.0019334.0
072169.0715160.0212671.0
180423.0357580.0412453.0
B
G
R
Z
Y
X
R G B
⋅
=
NTSC
NTSC
NTSC
B
G
R
Z
Y
X
111.1066.0000.0
114.0587.0299.0
200.0174.0607.0
⋅
−
−−
−
=
Z
Y
X
B
G
R
NTSC
NTSC
NTSC
902.0119.0058.0
028.0997.1985.0
288.0532.0909.1
⋅
−
−
−−
=
Z
Y
X
B
G
R
057311.1204043.0055648.0
041556.0875992.1969256.0
498535.0537150.1240479.3
709
709
709
9. November 3, 2015 Slide 9
400 500 600 700 800
400
0
1.0
2.0
-1.0
-2.0
b(λ)
g(λ)
r(λ)
700 nm
546.1nm
435.8 nm
r
g
b
λ0
λ0
P(λ)
λ
Color Matching of Monochromatic Light
r(λ),g(λ),b(λ) are CIE standard
r(λ) may be negative
Color Matching Functions
10. November 3, 2015 Slide 10
700 nm
546.1nm
435.8 nm
r
g
b
P(λ)
∫
∫
∫
=
=
=
λ
λ
λ
λλλ
λλλ
λλλ
dbPB
dgPG
drPR
t
t
t
)()(
)()(
)()(
P(λ)
λ
∫∫
∫∫
∫∫
=
=
=
λλ
λλ
λλ
λλλλλλ
λλλλλλ
λλλλλλ
dbPdbP
dgPdgP
drPdrP
tt
tt
tt
)()()()(
)()()()(
)()()()(
10
10
10
If
Then P0(l) and P1(l) are visually indistinguishable:
Metamerisim!!
Rt , Gt , Bt : tristimulus values
Rt may be negative
Color Matching of Ordinary Light
11. November 3, 2015 Slide 11
Rt Gt Bt to XYZ transformation
Rt may be negative for some of highly saturated (spectrally pure)
colors
RtGtBt color space is transformed to XYZ. The matrix coefficients are
carefully chosen so that XYZ values are all positive with other
convenient properties
=
t
t
t
B
G
R
Z
Y
X
5942.50565.00
0601.05906.41
1301.17517.17688.2
12. November 3, 2015 Slide 12
XYZ to xyz transformation
ZYX
Z
ZYX
Y
ZYX
X
z
y
x
++
++
++
=
=
=
Chromaticity Diagram
x
y
Visible colors
Displayable colors
Imaginary colors
13. November 3, 2015 Slide 13
LMS Cone Responses
−
=
Z
Y
X
S
M
L
100
0464.01834.12298.0
0787.06890.03897.0
−
=
Z
Y
X
S
M
L
9834.00136.00030.0
0061.06975.17036.0
1624.04296.07328.0
HPE CAT02
s(λ) m(λ)
l(λ)
0.4
0.2
0.6
0.8
1.0
700600500400 800
∫
∫
∫
=
=
=
λ
λ
λ
λλλ
λλλ
λλλ
dsPS
dmPM
dlPL
)()(
)()(
)()(
Unlike color matching functions, l(λ), m(λ), s(λ) are not standard
There is less confidence on l(λ), m(λ) and s(λ)
14. November 3, 2015 Slide 14
CIECAM02 Color Appearance Model
YCbCr RGBg RGB XYZ LMS LMSc
XYZ
Inverse
601
TF
Degamma
Tri-
Stimulus
TF
HPE Non-
linearity
J h s
Calculation
Chromatic
Adaptation
LMSh RGBap
Chromatic
Adaptation
TF (CAT02)
CAT02-1
LMSc J h s
Stimulus
Source
White Point
Source
Viewing Conditions
Source
Cone Response
Perception
Device Dependent Device
Independent
In implementation, five matrices
are combined into one
15. November 3, 2015 Slide 15
RGBg RGB XYZ LMS LMSc
XYZ
Display Gamma
Inverse Tri-
Stimulus
TF
HPE-1 Inverse Non-
linearity
RGBap
Calculation
Inverse
Chromatic
Adaptation
LMSh
RGBap
CAT02-1
CAT02
LMSc J h s
Cone Responses
Display
White Point
Display
Viewing Conditions
Inverse CIECAM02 Color Appearance Model
Perception
Device
IndependentDevice DependentDisplay
In TI’s implementation, five matrices
are combined into one
16. November 3, 2015 Slide 16
Non-linaerity
Brightness
Lightness
Opponent Colors
Hue
Saturation
Jhs Calculation ( Simplified Version)
))20/1(2( apapap BGRA ++=
m
wAAJ )/(=
)2)(9/1(
11/11/12
apapap
apapap
BGRb
BGRa
−+=
+−=
)/arctan( abh =
45.0
22
)20/21(
)(
kts
BGR
bahe
t
apapap
t
=
++
+
=
m<1. m increases as surround gets brighter
Aw: Brightness of white
Et(h): Eccentricity Factor
408.0
)( xxf =
17. November 3, 2015 Slide 17
Color processing
Contrast Enhancement
Saturation Enhancement
Hue Processing
Color Temperature (White Point) Management
Out-of-Gamut Management
• Perceptually impossible colors
• Colors that can not be displayed
18. November 3, 2015 Slide 18
Contrast enhancement: Manipulating of J channel to increase the
perceived dynamic range of the images
Two types of contrast enhancements: Global and Local
Global contrast enhancement:
Global tone curve based on frame-based histogram of J.
Local contrast:
Pixel-based locally adaptive tone curve
CIECAM02 attempts to preserve saturation to avoid washed-out look
Linear hue lines prevent hue shift
Contrast Enhancement
J J’
19. November 3, 2015 Slide 19
Saturation enhancement: Manipulating s channel to increase the
colorfulness of the images
Additional boost applied to memory color (Makes sky more blue and
grass more green) for more pleasing pictures
Gain suppression on skin to prevent over-saturating the skin (suntan).
Gain suppression in the dark area (low J) and near monochrome (low
color) to avoid enhancing the chroma noise.
Saturation Enhancement
Skin Tone
Dark Area
Low Color
s
h
J
a, b
s’
Memory
Colors
Gain Control
Gain
Calc
20. November 3, 2015 Slide 20
Tint adjustment
• Correct the hue shift mostly in legacy analog video,
specially NTSC composite video (Never The Same Color)
• Inaccurate color burst lock causes hue shift
Skin tone correction
• Shift the hue of colors closer to skin tones to the hue of
the “nominal” skin tone (Skin color that is considered to
most natural and beautiful)
Hue Processing
21. November 3, 2015 Slide 21
Gamut: Ranges of color that a display device can display
Source gamut usually does not match display gamut. We need
a gamut mapping strategy
Out-of-gamut conditions
• The original source colors being outside the display gamut
• Color processing that moves the colors out-of-gamut
Two strategies
• Maintaining color accuracy
• Expanding to fill display gamut
Gamut Mapping and
Out-of-gamut Management
22. November 3, 2015 Slide 22
1953 NTSC
HDTV/
Modern NTSC
Standard Gamut LCD Panel
23. November 3, 2015 Slide 23
1953 NTSC
HDTV/
Modern NTSC
Wide Gamut LCD Panel
24. November 3, 2015 Slide 24
1953 NTSC
HDTV/
Modern NTSC
Wide Gamut with Lasers
25. November 3, 2015 Slide 25
RGB
Source
Tristimulus
Matrix
XYZ
CIECAM02
Jhs Color
Processing
Inverse
CIECAM02
J’h’s’
J’h’s’ X’Y’Z’
Display
Inverse
Tristimulus
Matrix
R’G’B’
Display
Source
Maintaining Color Accuracy
Source
Display
Colors do not change if in-gamut. Clipped along
constant hue lines if out-of-gamut.
Advantage: Color accuracy
Disadvantage: Pictures looks dull. Not taking
advantage of full display gamut
Display
Source
26. November 3, 2015 Slide 26
RGB
Display
Tristimulus
Matrix
XYZ
CIECAM02
Jhs Color
Processing
Inverse
CIECAM02
J’h’s’
J’h’s’ X’Y’Z’
Display
Inverse
Tristimulus
Matrix
R’G’B’
Display
Source
Expanding to Fill Display Gamut
Display
Source
Source gamut is expanded to fill display gamut
Advantage: Vivid colors
Disadvantage: Slight hue shift
27. November 3, 2015 Slide 27
Out-of-gamut Management
Mapping out-of-gamut colors in-gamut
Clipping in RGB color space produces
hue shift. Clipping in Jhs color space
preserves hue.
(R,G,B)=
(0.5,0.5,0.9)
CIECAM02
(Source)
(J,h,s)=
(0.7,270,0.3)
Color
Processing
(J,h,s)=
(0.7,270,0.6)
Inverse
CIECAM02
Display
(R,G,B) =
(0.3,0.5,1.5)
Clipping
RGB
Cliiping s
Max s =
G(J,h)
Inverse
CIECAM02
(Display)
Hue shift
(J,h,s)=
(0.7,270,0.4) Less
saturated
(R,G,B) =
(0.3,0.5,1.0)
Gamut
Table
28. November 3, 2015 Slide 28
Gamut Table
The table has 1024 entries, organized as 2D RAM lookup tables
Gamut table defines the gamut boundary in Jhs color space. Incoming
s is clipped against the max s at the input of INV_CIECAM02.
Gamut table is computed off-line. The values depend on the
characteristic and the viewing conditions of the target display.
Multiple gamut tables may be required (e.g. one for each simulated
color temperature)
Gamut
Table
J
h
S in
Max_S
Min
S out
29. November 3, 2015 Slide 29
Gamut Table Visualization
90o
180o
270o
360o
S
30. November 3, 2015 Slide 30
Color Temperature (White Point)
Management
RGB XYZ LMS LMSc
Tri-
Stimulus
TF
Chromatic
Adaptation
TF (CAT02)
CEICAM02
RGB XYZ LMSInverse
Tri-
Stimulus
TF
Inverse
Chromatic
Adaptation
TF (CAT02)
Chromatic
Adaptation
Inverse
Chromatic
Adaptation
LMSc
Source
Display
Inverse
CEICAM02
White PointDegree of
Adaptation
White Point
Degree of
Adaptation
White Point:
Native and simulated
31. November 3, 2015 Slide 31
A R-G Y-B
+ +
+ +
-
+ +
+ -
L M S
S M L
700600500400 800
A
Y-B
R-G
0
0.5
1.0
-0.5
-1.0
0
0.4
0.2
0.6
0.8
1.0
LC
MC
SC
GL GM GS
Gs depend on degree of adaptation
and white point
Degree of adaptation increases as
brightness of display or surround increases
32. November 3, 2015 Slide 32
CIECAM02 is a powerful tool for color processing for TV application.
Provide a platform for systematic approach. Turn colors from art to science
Lab evaluation and Custom feedback have been positive
May find applications in digital imaging and other areas
Further optimization/simplification (e.g. gamut table)
xvYCC/Deep Color support
Ambient/background light adaptive color processing
Psychological hue manipulation (Intentional hue shift to make pictures more
pleasing. Extends the idea of skin tone correction)
Conclusion and Future Work
33. November 3, 2015 Slide 33
CIECAM02 Top-level Block Diagram
Part I
YCbCr-to-RGB
Conversion
DegammaYCbCr
RGB
with gamma
M0
RGB-to-XYZ
Coversion
Linear RGB XYZ
YCbCr-to-RGB Conversion:
R_gamma= GY*(y-64)+GVR*(Cr-512)
G_gamma= GY*(y-64)-GVG*(Cr-512) -GUG* (Cb-512)
B_gamma= GY*(y-64)+GUB*(Cb-512)
Degamma:
RGB = RGB_gamma*DGGL if RGB_gamma<DGTH
((RGB_gamma+DGOF)*DGS)^DGP if RGB_gamma>=DGTH
Matrix:
RGB-to-XYZ and XYZ-to-R’G’B’ combined to form
M00 M01 M02
M10 M11 M12
M20 M21 M22
34. November 3, 2015 Slide 34
CIECAM02 Top-level Block Diagram
Part II
M1
Programmable
3x3 Matrix
L1
ROM
Lookup Table
x^0.408
R'
G'
B'
M2
ROM
3x2 Matrix
Ra'
Ga'
Ba'
L2
Logic and
ROM
Lookup Table
Arctan ()
L3
Logic and
ROM
Lookup Table
Eccencrity
Factor
and Correction
Factor
h
|a|+|b|
a
b
X
M3
ROM
3x1 Matrix
/
A5
Et
L4
ROM
Lookup Table
t^0.45
t
A6
s
M4
ROM
Matrix
3x1
L5
Logic and
ROM
Lookup Table
log2()
A2
L6
ROM
Lookup Table
2^x
b/a
A2=cz
A6=(c*(1.64-0.29^n)^0.73/(Aw*FL^0.408*Nbb+0.04))^0.5A5=(420/1024)*Nc*Ncb
a b |a|>b| h
+ + Y theta
+ + N 90-theta
+ - Y 360-theta
+ - N 270+theta
- - Y 180+theta
- - N 270-theta
- + Y 180-theta
- + N 90+theta
theta = min(|a|,|b|)/max(|a|,|b|)
TABLE 1
See TABLE 1
M2=[[1,-12/11,1/11],[1/9,1/9,-2/9]]*99/16
M3=[1,1,21/20]*20/64
M4=[2,1,1/20]*20/64
JXYZ
Nbb omitted.
FL^0.408 omitted
A1
A AA
A1=1/Aw with Nbb
omitted
35. November 3, 2015 Slide 35
Inverse CIECAM02
Part I
L5^-1
Logic and
ROM lookup
table
2^x
1/A2
1/A6
L4^-1
Logic and
ROM lookup
table
x^(1/0.45)
t
S
5/207
60/253
X
L2^-1
Tan() X +x1 X
A11
X A/B
p2*t
X
+p1*t*(1/cos(h))
(Cb*tan(h)+Ca)*t
tan_hh
p1*t=(420/1024)*Nc*Ncb*Et
+1
Angle*
Convh St*theta
SW1*
Cross Switch
Ca
Cb
SW2*
Cross Switch
aa
bb
a
bL3
Logic and
ROM
Lookup Table
Eccencrity
Factor
A11=(420/1024)*Nc*Ncb
* SEE TABLE 2
L6^-1
Logic and ROM
lookup table
log2(x)
J
A
B
1/A2 =1/(c*z)
A10=Aw/Nbb
A10
AA
p2
St* x0
36. November 3, 2015 Slide 36
Inverse CIECAM02
Part II
M4
3x2
a
b
M4 = 1/1403* [[451,288],[-891,-261],[-220,-6300]]*16/99
64/61
+
p2
L1^-1Logic
and ROM
Lookup Table
x^(1/0.408)
M1^-1
h sa sb Ca Cb a b theta st s1t
0 -45 + + 5/207 60/253 aa bb h + +
45-90 + + 60/253 5/207 bb aa 90-h + +
90-135 - + 5/207 60/253 aa bb h-90 - +
135-180 - + 60/253 5/207 bb aa 180-h - -
180-225 - - 5/207 60/253 aa bb h-180 + -
225 -270 - - 60/253 5/207 bb aa 270-h + -
270 -315 + - 5/207 60/253 aa bb h-270 - -
315 -360 + - 60/253 5/207 bb aa 360-h - +
TABLE 2
XYZ
37. November 3, 2015 Slide 37
Inverse CIECAM02 Lite
Part III
Gamut Boundary
32X32X8
h
J
S Clipping Logic
MAX S(h,J)
J clipped
S clipped
38. November 3, 2015 Slide 38
Basic Facts About Colors
Lights are not the same as colors. Lights are physical entities
(photons, electromagnetic waves). Colors are perceptions ( like taste)
It takes at least three things for colors to happen: lights, object and
observer.
Lights can be directly measured (spectral power density). Colors are
exists only in the observers’ mind and can be described indirectly
(color matching)
Trichromacy: Most of common visible colors can be reproduced with
three carefully chosen primaries (RGB).
Not all sets of primaries are equal: Device dependent color primaries
(color space), device independent primaries. Same RGB values,
different colors
More information is needed in addition to RGB values to describe a
color precisely (lighting and viewing conditions, properties of the
primaries etc).
Trichromacy tells you how to make a color (recipe) but it does not tell
how the color looks like (tastes) .
More information is needed in addition to RGB values to describe a
color precisely (lighting and viewing conditions, properties of the
primaries etc).
We need a color model that better models and describes colors, in
other words how people perceive colors
39. November 3, 2015 Slide 39
400
r(λ)
g(λ)
b(λ)
P(λ)
P(λ)
λ
⋅
=
−
b
g
r
bbb
ggg
rrr
P
P
P
BGR
BGR
BGR
B
G
R
1
Where
Rr, Rg and Rb are the tristimulus values of r(λ)
Gr, Gg and Gb are the tristimulus values of g(λ)
Br, Bg and Bb are the tritimulus values of b(λ)
R,G and B are the intensity values of r(λ), g(λ) and b(λ)
40. November 3, 2015 Slide 40
Color Spaces Based on Primaries
1. Many color spaces exist based on different sets of primaries
sRGB (CIE Rec 709 HDTV)
1953 NTSC RGB
Modern NTSC
PAL/SECAM RGB
XYZ
YCbCr (Several varieties: xvYCC etc)
2. XYZ color space is of special interest. XYZ is based on theoretical
spectrally pure primaries (Primary consisting of only one single
frequency).
3. XYZ is the standard color space to describe a color in device-
independent manner. It is also used to define other color space (by
describing its three primaries in XYZ-called tristimulus values) by a 3x3
matrix.
⋅
=
709
709
709
950227.0119193.0019334.0
072169.0715160.0212671.0
180423.0357580.0412453.0
B
G
R
Z
Y
X
⋅
=
NTSC
NTSC
NTSC
B
G
R
Z
Y
X
111.1066.0000.0
114.0587.0299.0
200.0174.0607.0
R G B