1. Evaluation of Color Descriptors for Object and Scene Recognition Authors: Koen E. A. van de Sande, Theo Gevers, and Cees G. M. Snoek @ University of Amsterdam Presenter: Shao-Chuan Wang

2. Evaluation of Color Descriptors for Object and Scene Recognition Focus: Color features/descriptors on obj. and scene recognition Summary: The invariance of photometric transform aces and its effect on discriminative power. Conclusion: The usefulness of invariance is category-specific!

3. Photometric transforms (1/2) Light intensity scale invariant Light intensity shift invariant

4. Photometric transforms (1/2) Light intensity scale and shift invariant Light color change Light color change and shift

5. Color Descriptors (1/1) Histograms RGB, Hue, Saturation, rgHistogram, … Color Moments: contain spatial info. Color SIFT: combined color and SIFT HSV-SIFT, Hue-SIFT, …

6. Color Histograms (1/2) RGB-histogram Hue-histogram H and S are scale-invariant and shift-invariant w.r.t light intensity rg-histogram Scale-invariant Not shift-invariant Not that b is redudant Image from wikipedia

7. Color Histograms (2/2) Transformed color Scale and shift-invariant w.r.t light intensity. Opponent color histogram O1,O2 shift invariant O3: intensity, no invariant

8. Color SIFT Descriptors (1/3) HSV-SIFT SIFT over HSV channels Hue is unstable in gray axis Hue-SIFT (Van de Weijer 2006) Used Hue histogram weighing by its saturation Concatenate Hue histogram with SIFT Only SIFT is invariant; Hue histogram is not! (partial invariance) Hue Instability

9. Color SIFT Descriptors (2/3) OpponentSIFT SIFT over all channels in the opponent color space. Shift-invariant to light intensity. W-SIFT Eliminate O1 and O2’s intensity information Scale-invariant to light intensity rg-SIFT SIFT over r,g spaces Scale and shift invariant, but not invariant to light color changes/shifts

10. Color SIFT Descriptors (3/3) Transformed color SIFT SIFT over normalized transformed channels. Scale- and shift-invariant to light color changes and shift.

11. Experiments Implementation: Scale-invariants points are detected by Harris-Laplace point detectors Color descriptors are computed over the area around the points; all regions are proportionally re-sampled to a uniform 60x60 patch. Cluster descriptors with k = 40 (images) k = 4000 (video) SVM classifier with EMD/chi-square kernel

12. Benchmark (1/3) Image: PASCAL VOC 2007, over 20 object categories

13. Benchmark (2/3) Most objs were categorized better under scale- and shift- invariant to light intensity Some, such as car and dining table, do not benefit from such invariance.

14. Benchmark (3/3) Video: Mediamill Challenge, 39 object and scene categories

15. Evaluation of Color Descriptors for Object and Scene Recognition Conclusion: W-SIFT and rgSIFT, in general, outperform other color descriptors. Light intensity info. Is important for some categories Usefulness of invariance is category-specific.