This document proposes an approach for abnormal event detection in unseen scenarios using blob-level statistical analysis of video frames. The approach extracts frame-level features from foreground blobs using measures like blob area, count, and distance. These are transformed into temporal features considering speed and order. Event models are built offline using these features and an SVM classifier. During operation, only selected frame and temporal features are extracted in real-time, without additional training or tuning. Experiments on public safety and crowd datasets show the approach can detect events in unseen environments, outperforming optical flow-based methods.

1. Information Technology
Abnormal Event Detection in
Unseen Scenarios
Mahfuzul Haque and Manzur Murshed

2. Outline
 Event Detection for Public Safety
 Challenges
 Proposed Approach
 Experiments
 Summary
 Q&A
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
2

3. Event Detection for Public Safety
 Mob Violence
 Crowding
 Sudden Group Formation
 Sudden Group Deformation
 Shooting
 Panic Driven Behaviours
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
3

4. Event Detection
time




Specific event (e.g., run) detection vs. abnormality detection
An event persists for a certain duration of time
The duration is variable
The characteristics of the same event is
 variable in the same environment
 variable from one scene to other
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
4

5. Challenges
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
5

6. Challenges
Abnormal Event Detection
Supervised
Unsupervised
Manual Labelling,
Prior assumption of well
define event classes
No Event Model
Clustering of observed patterns,
Database of spatiotemporal patches
Semi-supervised
Normal event modelling:
manual labelling,
Abnormal event modelling:
unsupervised adaptation
Explicit Event Model
More Recent Approach
Mixture of Dynamic Bayesian Networks
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
6

7. Challenges
Build Event Model Once
Operate Everywhere
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
7

8. Proposed Approach
Build
•
•
•
•
Targeted Events
Extensive Feature Extraction
Feature Selection/Ranking
Supervised
Operation
•
•
•
•
•
No additional training
No parameter tuning
Selected feature extraction
No feature ranking
Real-time detection
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
8

9. Proposed Approach
f1
f2
Event
f3
Model
.
time
Frame-level Features
fn
Temporal Features
Classifier
 Event detection as temporal data classification problem
 A distinct set of temporal features can characterise an event
 Independent frame-level features extracted using blob statistical
analysis; no object / position specific information, no spatial
association
 Frame-level features are transformed into temporal features
considering speed and temporal order
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
9

10. Proposed Approach
Processes
Foreground
Detector
Frame-level
Feature Extractor
Temporal
Feature Extractor
Event
Models
Model Training (offline)
Frame-level
Feature Extraction
(30 features)
Background
Subtraction
Labelled frames
Temporal
Feature Extraction
(270 features)
Feature Ranking
and Selection
Event Model
Training
Foreground blobs
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
10

11. Blob-Statistical Analysis
Frame-level features








Blob Area (BA)
Filling Ratio (FR)
Aspect Ratio (AR)
Bounding Box Area (BBA)
Bounding box Width (BBW)
Bounding box Height (BBH)
Blob Count (BC)
Blob Distance (BD)
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
11

12. Blob Statistical Analysis
Blob Count (BC), Blob Area (BA)
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
12

13. Blob Statistical Analysis
Blob Distance (BD)
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
13

14. Blob Statistical Analysis
Aspect Ratio (AR)
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
14

15. Feature Extraction
Temporal features
2
1
4
3
6
5
Frame #
 Overlapping sliding window
 Temporal order
 Speed of variation
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
15

16. Feature Extraction
Top five features for four different events
Feature ranking using absolute value criteria of two sample t-test, based on
pooled variance estimate.
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
16

17. Proposed Approach
Model Training (offline)
Frame-level
Feature Extraction
(30 features)
Background
Subtraction
Labelled frames
Temporal
Feature Extraction
(270 features)
Feature Ranking
and Selection
Event Model
Training
Foreground blobs
Event Detection in the Operating Environment
Selective
Frame-level
Feature Extraction
Background
Subtraction
Incoming frames
Selective
Temporal
Feature Extraction
Trained
Event Models
Detection
Results
Foreground blobs
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
17

18. Summary and Discussion
Motion based approaches
 Key points detection
 Point matching in successive frames
 Flow vectors: position, direction, speed
Tracking based approaches
 Object detection
 Object matching in successive frames
 Trajectories: object paths
Common characteristics
 Inter-frame association
 Context specific information
 Event models are not generic
Hu et al. (ICPR 2008)
Xiang et al. (IJCV 2006)
Proposed approach
 No Inter-frame association
 Foreground blob detection
 Independent frame-level features =>
 Global frame-level descriptor based on
temporal features considering speed
blob statistical analysis, independent
and temporal order
of scene characteristics
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
18

19. Experiments
Model Training
•
•
•
•
•
•
•
Four different events: meet, split, runaway, and fight
CAVIAR dataset with labelled frames
80% of the test frames for model training
100 iterations of 10-fold cross validation
Remaining 20% of the test frames for testing
SVM classifier as event models
Separate model for each event
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
19

20. Experiments
Event Models
Unseen Scenarios in
Known Context
Unseen Scenarios in
Unknown Context
Greenfield
Outdoor
Abnormal Event Detection in Unseen Scenarios
Corridor
December 30, 2013
20

21. Experiments
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
21

22. Experiments
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
22

23. Experiments
• Abnormal event detection in unseen scenarios in
unknown context
• University of Minnesota crowd dataset (UMN dataset)
• The Runaway event model
• No additional training or tuning
• Three different sites
Greenfield
Outdoor
Corridor
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
23

24. Experiments
Abnormal Event Detection (UMN-9)
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
24

25. Experiments
Abnormal Event Detection (UMN-10)
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
25

26. Experiments
Abnormal Event Detection (UMN-01)
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
26

27. Experiments
Abnormal Event Detection (UMN-07)
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
27

28. Experiments
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
28

29. Experiments
Method
AUC
Proposed Method
0.89
Pure Optical Flow [1]
0.84
[1] R. Mehran, A. Oyama, and M. Shah, “Abnormal crowd behavior detection using social force model,” in Proc. IEEE
Conference on Computer Vision and Pattern Recognition CVPR 2009, Event Detection2009, pp.Scenarios
Abnormal 20–25 June in Unseen 935–942. December 30, 2013
29

30. Q&A
Abnormal Event Detection in Unseen Scenarios
December 30, 2013
30