ICPR 2012

Foreground Detection via Robust Low Rank Matrix
Factorization including Spatial Constraint with Iterative
Reweighted Regression

C. Guyon, T. Bouwmans and E. Zahzah
charles.guyon@univ-lr.fr

MIA Laboratory (Mathematics Images & Applications), University of La Rochelle, France
Presenter: Muriel Visani (L3i lab - University of la Rochelle)
—
ICPR2012, Tsukuba, Japan

November 14, 2012

C. Guyon, T. Bouwmans and E. Zahzah charles.guyon@univ-lr.fr via Robust Low Rank Matrix Factorization including / 25 U
Foreground Detection (MIA Laboratory (Mathematics Images2012
November 14, & Applications),
1 Spatia

Summary

1 Introduction and motivation on IRLS

2 Temporal constraint with an adapted norm

3 Diagram ﬂow and spatial constraint

4 Experimental Results

5 Conclusion

2 Spatia

Introduction and motivation

Purpose
Foreground detection : Segmentation of moving objects in video sequence
acquired by a ﬁxed camera.
Background modeling : Modelization of all that is not moving object.

Involved applications
Surveillance camera
Motion capture

On the importance
Crucial Task : Often the ﬁrst step of a full video surveillance system.

Strategy used
Eigenbackground decomposition.
3 Spatia

Eigenbackgrounds
Find an « ideal » subspace of the video sequence, which describes the best
as possible the (dynamic) background.

Fig.1 The common process of background subtraction via PCA (Principal Component
Analysis). At the ﬁnal step, an adaptative threshold is used to get a binary image.

Without a robust framework, the moving object may be absorbed in the model !
4 Spatia

Data Structure Transformation
First, we consider a video sequence as a matrix A ∈ Rn×m

n is the amount of pixels in a frame (∼ 106 )
m is the number of frames considered (∼ 200)
5 Spatia

IRLS : Vector version (1)

The usual IRLS (Iteratively Reweighted Least Squares) scheme for solve
argmin ||Ax − b||α is given by :
x

D (i) = diag((ε + |b − Ax (i) |)α−2 )
(1)
x (i+1) = (At D (i) A)−1 At D (i) b

This IRLS method is convergent for 1 ≤ α < 3. An more suitable
formulation is :

r (i) = b − Ax (i)
D = diag((ε + |r (i) |)α−2 )
(2)
y (i) = (A DA)−1 A Dr (i)
x (i+1) = x (i) + (1 + λopt )y (i)
for λopt computed in a second inner loop. It is convergent for 1 ≤ α < +∞

6 Spatia

IRLS : Vector version (2)

For spatio/temporal RPCA, it needs to solve the following general problem :

argmin ||Ax − b||α + λ||Cx − d||β (3)
x

By derivation, the associated IRLS scheme is,
r1 = b − Ax (i) , r2 = d − Cx (i) , e1 = ε + |r1 |, e2 = ε + |r2 |
α 1 β 1 −1 β−2
D1 = ( e1 ) α −1 diag(e1 ), D2 = λ( e2 ) β diag(e2 )
α−2
(i) −1
(4)
y = (A D1 A + C D2 C ) (A D1 r1 + C D2 r2 )
x (i+1) = x (i) + (1 + λopt )y (i)

Good news : Just few lines in Matlab !

7 Spatia

IRLS : Matrix Version

More generally, we consider the following matrix regression problem with
two parameters norm (α, β) and a weighted matrix W ,
n m
α 1
min ||AX − B||α,β with ||Mij ||α,β = ( ( Wij |Mij |β ) β ) α (5)
X W W
i=1 j=1

The problem is solved in the same manner on matrices with a reweighted
regression strategy,
Until X is stable, repeat on each k-column
R ← B − AX
S ← ε + |R| (6)
α −1
β−2 β
Dk ← diag(Sik ◦ ( j (Sij ◦ Wij )) β ◦ Wik )k
Xik ← Xik +(1+Λ(max(α, β)))(At Dk A)−1 At Dk Rik

8 Spatia

Various RPCA formulation (only for α = 1)
PCA with a ﬁxed rank is : min ||S||F
L,S
s.t. Rank(L) = k (7)
A=L+S
R(obust)PCA is (Non convex and NP-hard ) :
min ||σ(L)||0 + λ||S||0
L,S (8)
s.t. A=L+S
Convex relaxed problem of (8) is RPCA-PCP proposed by Candès et al. [1] :
min ||σ(L)||1 + λ||S||1
L,S (9)
s.t. A=L+S
where σ(L) means singular values of L.
A mix is Stable PCP of Zhou et al. [2] (both entry-wise and sparse noise) :
min ||σ(L)||1 + λ||S||1
L,S (10)
s.t. ||A − L − S||F < δ

All of them could be solved by Augmented Lagrangian Multipliers (ALM).
9 Spatia

Video examples

Some examples, temporal RPCA and ideal RPCA with ground truth ﬁtting.

Foreground Detection (MIA Laboratory (Mathematics Images & Applications),
November 14, 2012 10 Spatia

Summary





5 Conclusion


Sparse solution
In RPCA, residual error is sparse.
Using the RPCA decomposition on a synthetic low-rank random matrix
plus noise, the error looks like :

Same principle with video. Sparse noise (or outliers) are the moving objects.

Let’s play with norms

Varying the α, β norm → Diﬀerent kind of recovering pattern error.


Let’s play with norms...(2)

Some issues
What is the best speciﬁc norm for temporal constrain ?
Initial assumption is ||.||2,1 . Conﬁrmed experimentally ?


Validation
If ideal eigenbakgrounds are that, best norm should be ...

Let us denote Lopt , the ideal low-rank subspace which outliers do not contribute to PCA

Experimental validation

Let us denote Lα,β , the low-rank
recovered matrix with a ||.||α,β -PCA.
The plot shows the error between
||Lopt − Lα,β ||F for parameters α
and β chosen freely. The darkest
value means that the error is the
smallest here.

||S||2,1 is not optimal, but for convenience we use it.
The benefit of the ad hoc block-sparse hypothesis is confirmed by
testing its efficiency directly on video dataset.

Experimentation done on dynamic category of dataset change detection
workshop 2012 : http://www.changedetection.net/


Summary





5 Conclusion


Overview & addition of a spatial constraint via TV

Figure: Overview of the learning and evaluation process. Learning process needs
GT (Ground Truth) for better ﬁts the eigenbackground components.

Spatial Constraint via TV
Suppose A = L + S where L and S are computed via some kind of
RPCA techniques with the addition of Total Variation penalty on S.
This penalty increases connected (or connexe) shapes.


Exemple with a synthetic 1-D signal


Summary





5 Conclusion


Experimental Protocol
RPCA-IRLS is compared for the following four recent robust methods :
Low-Rank Block sparse Decomposition (LBD, 2011) [3]
Low-Rank Representation (LRR, 2011) [4]
Symmetric Alternating Direction Augmented Lagrangian (SADAL, 2011) [5]
Grassmannian Robust Adaptive Subspace Tracking Algorithm (GRASTA, 2012) [6]

References
[1] E. Candes, X. Li, Y. Ma, and J. Wright, Robust principal component analysis, International Journal of ACM,
vol. 58, no. 3, May 2011.
[2] Z. Zhou, X. Li, J. Wright, E. Candes, and Y. Ma, Stable principal component pursuit,IEEE ISIT
Proceedings, pp. 1518-1522, Jun. 2010.
[3] G. Tang and A. Nehorai, Robust principal component analysis based on low-rank and block-sparse matrix
decomposition, CISS 2011, 2011.
[4] Z. Lin, R. Liu, and Z. Su. Linearized alternating direction method with adaptive penalty for low-rank
representation. NIPS 2011, Dec. 2011.
[5] S. Ma. Algorithms for sparse and low-rank optimization : Convergence, complexity and applications. Thesis,
2011.
[6] J. He, L. Balzano, and A. Szlam. Incremental gradient on the grassmannian for online foreground and
background separation in subsampled video. Conference on Computer Vision and Pattern Recognition
(CVPR), June 2012.


Experimental Protocol & Quantitative Results

Optimal threshold is chosen for maximizing F-measure criterion
which is based 2 × 2 histogram of True/false/positive/negative :

TP TP 2 DR Prec
DR = , Prec = , F =
TP + FN TP + FP DR + Prec

Good performance is then obtained when the F-measure is closed to 1
Time consumption is not take into account in the evaluation process.

Figure: F-Measure on the Wallﬂower and I2R dataset.


Quantitative Results

Here, we show other experimental results on the real dataset of BMC 2012,

Video Recall Precision F-measure PSNR Visual Results
1 0.9139 0.7170 0.8036 38.2425
2 0.8785 0.8656 0.8720 26.7721
3 0.9658 0.8120 0.8822 37.7053
4 0.9550 0.7187 0.8202 39.3699
5 0.9102 0.5589 0.6925 30.5876
6 0.9002 0.7727 0.8316 29.9994
7 0.9116 0.8401 0.8744 26.8350
8 0.8651 0.6710 0.7558 30.5040
9 0.9309 0.8239 0.8741 55.1163

Table: Quantitative results with common criterions. Last column : sample of the
original video, GT and our results of the ﬁrst four real video sequences.


Summary





5 Conclusion


Conclusion

Advantages
Experiments on video surveillance datasets show that this approach is more
robust than other recent RPCA formulation in presence of dynamic
backgrounds (DC) and illumination changes (IC).
Well suited for video with spatially spread and temporarily sparse outliers.

Disadvantages
Small local motions, like « waving trees » are not (yet) well modelized by
this kind of global PCA. For example, IC needs few eigenBackground and
DC needs more with the risk to integrate moving objects into the model.

Future Works
Lack in computation time : Further research consists in developping an
incremental version to update the model at every frame and to achieve the
real-time requirements.

ICPR 2012

Recomendados

Recomendados

Más contenido relacionado

La actualidad más candente

La actualidad más candente (20)

Destacado

Destacado (20)

Similar a ICPR 2012

Similar a ICPR 2012 (20)

Último

Último (20)

ICPR 2012