Contribution of Non-Scrambled Chroma Information in Privacy-Protected Face Images to Privacy Leakage. Presentation given at the 10th International Workshop on Digital Forensics and Watermarking (IWDW'11).
Note that a more extensive objective and subjective study of privacy protection in video surveillance systems can be found in the following book chapter:
H. Sohn, D. Lee, W. De Neve, K.N. Plataniotis, and Y.M. Ro. An objective and subjective evaluation of content-based privacy protection of face images in video surveillance systems using JPEG XR. Effective Surveillance for Homeland Security: Balancing Technology and Social Issues. CRC Press / Taylor & Francis. May 2013. pp. 111-140.
http://www.citeulike.org/user/wmdeneve/article/10831550
http://www.crcpress.com/product/isbn/9781439883242
Biography Of Angeliki Cooney | Senior Vice President Life Sciences | Albany, ...
Contribution of Non-Scrambled Chroma Information in Privacy-Protected Face Images to Privacy Leakage
1. Contribution of Non-Scrambled Chroma Information in
Privacy-Protected Face Images to Privacy Leakage
10th International Workshop on Digital Forensics and Watermarking
October 2011
Hosik Sohn1, Dohyoung Lee2, Wesley De Neve1,
Konstantinos N. Plataniotis2, and Yong Man Ro1
1Korea Advanced Institute of Science and Technology (KAIST),
Image and Video Systems Lab
2University of Toronto,
Multimedia Lab
2. -2-
Contents
1. Introduction
2. Layered Scrambling for Motion JPEG XR
3. Assessment of Chroma-induced Privacy Leakage
3.1 Objective Assessments
3.2 Subjective Assessments
4. Discussion and Conclusions
4. -4-
1. Introduction
Present-day video surveillance systems often come with high-speed network
connections, plenty of storage capacity, and high processing power
The increasing ability of video surveillance systems to identify people has
recently raised several privacy concerns
To mitigate these privacy concerns, scrambling can be leveraged to conceal
the identity of face images in video content originating from surveillance
cameras
Privacy protected surveillance videos
5. -5-
1. Introduction
The past few years have witnessed the development of a wide range of
content-based tools for protecting privacy in video surveillance systems
Dependent on the location where scrambling (or encryption) is applied, three
different approaches of scrambling can be distinguished
Uncompressed domain scrambling
Transform domain scrambling
Compressed bit stream domain scrambling
One of the main challenges is the concealment of privacy-sensitive regions by
making use of invertible transformation of visual information at a low
computational cost
6. -6-
1. Introduction
Content-based tools for privacy protection need to find a proper balance
between the level of security offered and the amount of bit rate overhead
In general, altering the visual information present in privacy-sensitive regions
typically breaks the effectiveness of coding tools
Coding
Security
efficiency
level
To limit the bit rate overhead, many content-based tools for privacy protection
only scramble luma information, leaving chroma information unprotected
7. -7-
1. Introduction
In this paper, we investigate the contribution of non-scrambled chroma
information to privacy leakage
To that end, we study and quantify the influence of the presence of non-
scrambled chroma information on the effectiveness of automatic and human
FR
Objective assessment: we apply automatic FR techniques to face images that
have been privacy-protected in the luma domain
Subjective assessment: we investigate whether agreement exists between the
judgments of 32 human observers and the output of automatic FR
8. -8-
1. Introduction
FR vs. perception-based security metrics for assessing the level of privacy
Luminance Similarity Score (LSS), Edge Similarity Score (ESS), and Local
Feature-based Visual Security Metric (LFVSM)[1,2]
Note that these metrics are general in nature and are thus not able to take
advantage of domain-specific information (e.g., face information)
[1] Tong, L., Dai, F., Zhang, Y., Li, J. “Visual security evaluation for video encryption,” in: Proceedings of ACM International Conference on
Multimedia, 835–838 (2010)
[2] Mao Y., Wu M., "A joint signal processing and cryptographic approach to multimedia encryption," IEEE Transactions on Image Processing,
15(7), 2061-2075 (2006)
10. -10-
2. Layered Scrambling for Motion JPEG XR
The video surveillance system studied makes use of Motion JPEG XR to
encode surveillance video content
Motion JPEG XR offers a low-complexity solution for the intra coding of high-resolution
video content, while at the same time offering quality and spatial scalability provisions
Layered scrambling for JPEG XR [3]
Modified JPEG XR encoder
Secret key
DC subband
Scrambling
LBT LBT Q Pred.
(RLS) • Adaptive
LP subband entropy
Adaptive Scrambling coding
Q Pred.
scan (RP) • Fixed
HP subband/Flexbits length
Adaptive Scrambling coding
Q Pred.
scan (RSI)
[3] Sohn, H., De Neve, W., Ro, Y.M., “Privacy Protection in Video Surveillance Systems: Analysis of Subband-Adaptive Scrambling in JPEG
XR,” IEEE Transactions on Circuits and Systems for Video Technology, 21, 170–177 (2011)
11. -11-
2. Layered Scrambling for Motion JPEG XR
Overview of the layered scrambling technique
- Random level shift (RLS) for DC subbands
DCcoeff e DCcoeff R(L),
- Random permutation (RP) for LP subbands
LPcoeffi e LPcoeff j , where i 1,..., C, j x1 ,..., xC ,
- Random sign inversion (RSI) for HP subbands
HPcoeff , if r 1
HPcoeff e ,
HPcoeff , otherwise
N denotes the number of MBs, L denotes the RLS parameter, K denotes the number
of non-zero LP coefficients in a MB, and M denotes the number of non-zero HP
coefficients in a MB
13. -13-
3.1. Objective Assessments
Experimental setup
FR techniques used: PCA, FLDA, LBP
Face images: 3070 frontal face images of 68 subjects from CMU PIE
(68 gallery, 340 training, and 2662 probe face images)
Probe face images represent privacy-protected face images that appear in video
content originating from surveillance cameras
Performance evaluation: Cumulative Match Characteristic (CMC) curve
Notations
Notation Explanation
DC, LP, and HP DC, LP, and HP subband
S3 DC+LP+HP
S2 DC+LP
S1 DC
Subscripts (Y, Co, Cg) Luma and chroma channels (Y, Co, and Cg)
Prime (′) Scrambled image data
14. -14-
3.1. Objective Assessments
Influence of distance measurement on FR effectiveness
Distance metric: Euclidean, Mahalanobis, Cosine, and Chi-square distance
DE : Euclidean distance
DM : Mahalanobis distance
DC : Cosine distance
DH : Chi-square distance
In the remainder of our experiments, we make use of the Euclidean distance metric for
PCA- and FLDA-based FR, and the Chi-square distance metric for LBP-based FR
15. -15-
3.1. Objective Assessments
Scrambled luma information
Assumes that an adversary is not able to take advantage of the possible presence of
non-scrambled chroma information in the privacy-protected probe face images
16. -16-
3.1. Objective Assessments
Scrambled luma and non-scrambled chroma information
We investigate whether layered scrambling is still effective when the
scrambled luma channel and the non-scrambled chroma channels are
simultaneously used for the purpose of automatic FR
We assume that an adversary has access to the compressed bit stream
structure, and thus to the non-scrambled chroma information
We adopt feature-level fusion in order to take advantage of non-
scrambled chroma information
19. -19-
3.2. Subjective Assessments
Experimental setup
Number of observers: 32
We presented three scrambled probe face images of different subjects to the
human observers for each experimental condition
Assessment method
Human observers were asked to select the gallery face image that is most similar to
the given probe face image
Human observers were also able to study the probe face images at different zoom
levels
Gallery face images used for the subjective assessments
23. -23-
4. Discussion
For video surveillance applications requiring a high level of privacy protection, both
the luma and the chroma channels need to be scrambled
At the cost of a higher bit rate overhead
Layered scrambling for both the luma (Y) and the chroma channels (Co and Cg)
24. -24-
4. Discussion
Bit rate overhead
Security (ideal case)
Sub-sampling decreases the level of privacy protection, given the lesser amount
of data available for scrambling
Total number of combinations required to break the protection of 10 MBs is
reduced from 3.6×10722 (4:4:4) to 1.7×10360 (4:2:0)
25. -25-
5. Conclusions and Future Work
This paper studied and quantified the influence of non-scrambled chroma infor-
mation on the effectiveness of automatic and human FR
Our results show that, when an adversary has access to the coded bit stream
structure, the presence of non-scrambled chroma information may significantly
contribute to privacy leakage
For video surveillance applications requiring a high level of privacy protection, our
results indicate that both luma and chroma information needs to be scrambled at the
cost of an increase in bit rate overhead
In order to compile a benchmark for privacy protection tools, future research will
focus on identifying additional worst case scenarios
27. -27-
APPENDIX A
Effectiveness of general-purpose visual security metrics
Visual security metrics used
Luminance Similarity Score (LSS), Edge Similarity Score (ESS), and Local Feature-
based Visual Security Metric (LFVSM)
The lower the values computed by the visual security metrics, the higher the
visual security
28. -28-
APPENDIX B
Paper of interest
Sohn, H., De Neve, W., Ro, Y.M., “Privacy Protection in Video Surveillance
Systems: Analysis of Subband-Adaptive Scrambling in JPEG XR,” IEEE
Transactions on Circuits and Systems for Video Technology, 21, 170–177
(2011)
Book chapter of interest
Sohn, H., Lee, D., De Neve, W., Plataniotis, K.N., Ro, Y.M., “An objective and
subjective evaluation of content-based privacy protection of face images in
video surveillance systems using JPEG XR,” Accepted for publication in
Effective Surveillance for Homeland Security: Balancing Technology and
Social Issues, CRC Press / Taylor & Francis, To be published in 2013
IVY Lab video surveillance data set
http://ivylab.kaist.ac.kr/demo/vs/dataset.htm