This document discusses intelligent video surveillance and sousveillance. It covers topics such as video surveillance market trends, important crime cases solved using CCTV footage, and technology used in intelligent video surveillance systems. Computer vision algorithms are used to add intelligence to video surveillance, going beyond just monitoring to visual surveillance. The document also presents examples of intelligent surveillance applications and research from universities and companies.
1. 本著作採用創用CC 「姓名標示」授權條款台灣3.0版
Intelligent
Video Surveillance and
Sousveillance
Wang, Yuan-Kai(王元凱)
Electrical Engineering Department,
Fu Jen Univ., Taiwan (輔仁大學電機工程系)
Email: ykwang@mail.fju.edu.tw
URL: http://www.ykwang.tw
2010/11/06
2. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 2
Contents
1. Image, Vision and Intelligence
2. Intelligent Video Surveillance
3. Intelligent Video Sousveillance
4. Conclusions
3. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 3
1. Image, Vision, and
Intelligence
Human's vision system
4. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 4
Image, Vision, and
Intelligence
in the Digital World
Digital vision system
+
Robust Computer Vision Algorithms
5. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 5
Camera Based Environment
Person : Camera = 1 : N
Outside-In : Video Surveillance, Human Computer Interface
Inside-Out : Video Sousveillance, Egocentric Vision
6. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 6
2. Video Surveillance
Video surveillance
Use video camera to monitor an
area for crime investigation
From video surveillance
to visual surveillance
Impose video analytics
by computer vision algorithm
7. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 7
Video Surveillance Market
• CCTV has been a mass-product
market
• Since the 911 event, the market
is continuously increasing
(百萬美元)
Source: JP Freeman
8. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 8
Who’s Watching You?
UK has the most CCTV cameras
in Europe
4.2 million cameras which is
20% of the world's CCTV
1 camera for every 14 people in UK
On average a person can be
caught on camera 200-300 times a
day
11. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 11
Important Crime Cases
近年來運用路口監視器
偵破社會矚目重大案件
白米炸彈客
汐止市殺警奪槍案
蠻牛千面人案
台南國道襲警奪槍案
新莊襲警奪槍案
英國倫敦地鐵爆炸案
台中角頭槍殺案
12. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 12
Case Study
94年5月17日台中市蠻牛千面人案,
造成全省恐慌
破案關鍵在於
幾個放置毒蠻牛
的超商監視器
錄到千面人身影
歹徒車號被
提款機監視器
清楚拍下
動員500警員
觀看6000小時的錄影資料
13. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 13
CCTV Video Surveillance
Video Display & Record
VCR / DVR
Analog
Multiplexer components
Centralized
Monitoring
Video Capture
analogue analogue analogue analogue
14. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 14
Digital Video Surveillance
High scalibility
IPCam + analog camera
Network transmission
Remote control
Digital storgage
digital
Network
digital
digital
analogue analogue analogue
analogue
15. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 15
Technology Trend of
Video Surveillance
資料來源:拓璞產業研究所,2008年5月
16. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 16
Surveillance over IP
Paradigm shift of video surveillance
Role from security monitoring to the personalized video contents
Advent of the intelligent surveillance
Changes in technology & desire
1. Network
2. Video compression
3. Live images Intelligent
Surveillance
IP Surveillance
CCTV (DVR)
CCTV (VCR)
1G
2G 3G
17. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 17
Why Intelligent Surveillance
Too many cameras, too few human guards
“After only 20 minutes, human attention to
video monitors degenerates to an
unacceptable level.” (Sandia National Laboratories)
18. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 18
Applications of
Visual Surveillance
19. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 19
Visual Surveillance
Visual Surveillance = Digital CCTV + Video Analytics
Smart/Intelligent Surveillance
20. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 20
Video Analytics
影像擷取 相機異常偵測 人臉辨識 查詢、過濾、聯防
Video Image Object Object Object Behavior
Capture Enhance /Event Tracking
/Event
Analysis Retrieval
Detection Recognition
強光抑制 警戒線、
跌倒、人潮行為分析
路徑追蹤、流浪漢監控
21. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 21
IBM S3
Exploratory Computer Vision Group in IBM T.J. Watson Research Center.
http://www.research.ibm.com/ecvg/
24. 王元凱 以視覺為基礎之智慧型環境(Vision Based Intelligent Environment) p. 24
經濟部學界科專計畫 (1/3)
MOEA’s Technology Development
Program for Academia
Name: Construction of Vision-
Based Intelligent Environment
Phase I : 2004/5 ~ 2008/4
Phase II: 2008/11 ~ 2012/10
Involved people
29 professors from 18 universities
110 staffs
Fu Jen University Department of Electronic Engineering Yuan-Kai Wang
25. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 25
經濟部學界科專計畫 (2/3)
智慧型建築
目標:開發智慧型建築內部空間不可或缺的全
方位、主動式、機動性的智慧性視訊監控系統
A1 日夜活動式廣域安全監視
系統
A2 視訊監控中央管理系統
A3 室內突發事件分析系統
攝影機網路 感測網路
26. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 26
經濟部學界科專計畫 (3/3)
智慧型社區與城市
目標:開發戶外社區及城市大範圍區域之穩定、成熟而
具產品面向的智慧性視覺監控系統
B1 人車偵測與辨識系統
B2 都會區人物追蹤系統
B3 室外事件分析與搜尋系統
27. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 27
Activities in the VBIE Project
參加國際展覽
研究技術需
展示化:技術需能常駐展示
系統化:大型整合展示
指標化:技術有量化指標
市場化:建立產業鏈地圖、政策規劃
專利化:專利佈局、專利地圖分析
商品化:網路行銷百餘項技術
參與國際標準制訂(ONVIF)
引導業界投資
與警政機關合作
28. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 28
A Proposed Architecture for
Police Office (1/2)
智慧型視訊監控技術在警政治安上之可行性研究,
詹毓青,中央警察大學資訊管理所碩士論文,2009
29. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 29
A Proposed Architecture for
Police Office (2/2)
智慧型視訊監控技術在警政治安上之可行性研究,
詹毓青,中央警察大學資訊管理所碩士論文,2009
30. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 30
Demos from ISLab@FJU
2.1 Mobile Video Surveillance
2.2 Night Vision
2.3 Reconfigurable Hardware for
Moving Object Detection
2.4 Super-resolution
2.5 Camera Tampering Detection
2.6 System Integration
31. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 31
2.1 Mobil Video Surveillance
Event-driven Instant Messaging
32. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 32
Current Mobile Surveillance
Mobile phone directly connects to
the camera
Mobile networks
(circuit-switched
& packet-switched) Remote monitor
Camera client
Drawbacks:
1. Not event-driven
2. High communication cost
33. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 33
Instant Alarm Messaging
Is Important
34. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 34
Our System Architecture
Keyframe Web
Selection Server
PC
Object
1. Object Info. Video
Detection
2. Keyframe Streaming
&Tracking
3. Video Clip Mobile
Phone
Video SMS
Transcoding Messaging
35. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 35
Our System: Web-based
Browsing Interface
36. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 36
Our System –
Event, Key Frame, Video Clip
37. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 37
Mobile Interface
SMS Message for Mobile
Event Notification video streaming
39. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 39
Performance
Instant messaging
PC: Within 1 seconds
Mobile phone: ~ 5 seconds
上午 12:52 - 上午 12:00
Average time of an event
3.9 sec
上午 12:00
Face detection
Skin detection
3.4 sec
1.7 sec
下午 11:45 - 上午 12:00
間隔描述
上午 11:54 - 下午上午 11:31 - 下午 2:24
2:02 上午 11:40 - 下午 2:15
Detection & Key frame MMS
tracking Selection 3~5 sec
5.7 sec 5.1 sec
Mobile Streaming
(RTSP)
上午 12:52 - 上午 12:00
Transcoding
4.1 sec
40. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 40
2.2 Night Vision
Night vision means to detect
moving objects from night images
< 1 lux
High-level noises
Night vision should be important
because crimes
usually happens
at night
41. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 41
Moving Object Detection
Background subtraction is the
most important method
However, it
can not work
at night
Bad image
quality
Bad detection
quality
42. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 42
An Example Environment
Entry of elevators in the hall
Two challenges
Low-light-level:
Low contrast, strong noise
Drastic light change:
When elevators open, strong light emits
from the elevators 緊急出口警示燈 (微弱)
現場唯一光源(門外屋簷) IPCam架設位置
1 2 3 4
真實場景 by RICOH RX200 (Defaults Setting)
44. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 44
2.3 Reconfigurable Hardware
for Moving Object Detection
Background subtraction, ...
• 2.8 GHz Intel CPU
• Software: C/C++ FPGA
• Frame rate: 10 fps for 1 channel
45. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 45
Background Subtraction
Current
Frame B k + 1
Background
M k +1 ( x, y ) P k+1 Image Update
Background Image
= Pk +1 ( x, y ) − Bk ( x, y ) -
Bk
M
k + 1
Bk +1 ( x, y )
= αBk ( x, y ) + (1 − α ) Pk +1 ( x, y )
Post Processing
Motion Object Image
Speed up by (1) Circuit design, (2) Parallelization
46. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 46
Parallelization by Hardware
Parallelism: 7-level pipeline
SIMD with stream processing
47. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 47
Design & Results
Hardware: Altera Cyclone II 2C35
Design: Verilog HDL with Quartus II
Background New Frame Result
Frame rate
• Background module : 368 fps
• Whole system : 51 fps
48. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 48
Experimental Comparison
PC: 2.8GHz CPU, C implementation
FPGA can speed up 500 times
2.8G
51
CPU
FPGA
25M 10
Clock(Hz) FPS
49. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 49
2.4 Super-resolution
Object images are too small
Super-resolution is to
Construct a high-resolution image
from low-resolution images
Challenges: Ill-posed problem
Low-resolution High-resolution
image image
50. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 50
The Proposed Method
Pervious methods
Interpolation, reconstruction
Feature-based learning
Dimension Expansion
Y X
Image Space
Dimension • Maximum A Posteriori
Reduction
(MAP) Estimator
Feature Space Dimension
• Gibbs Prior
Expansion
x • 2D2 Locality
High-resolution Preserving Projection
Features
51. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 51
Experiments
Four-time scale
Low-resolution is 70×50
High-resolution is 140×100
Ground Truth Low-resolution Bicubic Our Method
NN interpolation interpolation
52. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 52
Experimental Results
53. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 53
2.5 Camera Tampering
Detection
Possible tampering
Spray-painting
Replacement
Hit/collision
Defocus
Blocking
...
54. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 54
Tampering Examples (1/2)
Before After
Spray
painting
Defocus
55. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 55
Tampering Examples (2/2)
Before After
Replacement
(Day)
Replacement
(Night)
56. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 56
The System
Sabotage
detection
before visual
surveillance
algorithms
Server-based
solution for
large-scale
surveillance
systems
57. 王元凱 以視覺為基礎之智慧型環境(Vision Based Intelligent Environment) p. 57
2.6 System Integration
in the VBIE Project
Smart Building
Integrated monitoring within building
Tracking target: person
Smart Campus
Long range tracking within campus
Tracking target: car and person
Fu Jen University Department of Electronic Engineering Yuan-Kai Wang
58. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 58
Heterogeneous Cameras
We use various kinds of cameras
環場攝影機
PTZ攝影機
紅外線熱像攝影機
固定式攝影機
活動攝影機
活動攝影機畫面
59. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 59
Smart Building (1/2)
Characteristics
11 techniques are integrated by top-
down design
It follows the CMMI software
engineering method
Documents of spec requirement,
testing, ...
A long-term test site is built
60. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 60
Smart Building (2/2)
NTSC一般攝影機 PTZ網路攝影機 魚眼攝影機
62. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 62
Smart Campus (2/2)
63. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 63
3. Video Sousveillance
Environment cameras
Outside-in images
Body-worn cameras
Inside-out images
64. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 64
Paradigm Shift
Evolution of computing paradigm
Desktop computing
Mobile computing
Wearable computing
Evolution of camera technology
Desktop vision
Mobile vision
Wearable vision⇒ Egocentric vision
65. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 65
Desktop Computing
No interaction with
the environment
User is focused
on the system
stationary
66. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 66
Jurassic Mobile Computing
Goal: Stay in touch for mobile usage
67. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 67
Current Mobile Computing
Still has drawbacks
Hands are not free
Screen is too small
Need keyboard
Not proactive
68. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 68
Wearable Computing
Wear the computer on the body
Smart glasses
Smart clothing
69. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 69
The Hardware
Computer: Embedded system
Input
No keyboard
& mouse Gestures
But sensors
Output Text Audio Special Purpose
70. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 70
Input Sensor - Glove
Hand Gesture Interaction
71. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 71
Output Display
Optical see-through HMD (Head
Mounted Display)
72. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 72
Add Vision Sensor
Video see-through HMD
Wearable Vision
73. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 73
Why Wearable Vision (1/2)
Capture of casual events
74. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 74
Why Wearable Vision (2/2)
Game assistant
75. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 75
Where to Wear an Extra Eye
Mayol-Cuevas, Tordoff, Murray, "On the Choice and Placement of
Wearable Vision Sensors," IEEE Trans. SMC A, March 2009.
76. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 76
Where to Wear an Extra Eye
77. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 77
Steve Mann’s WearComp
S. Mann, "Humanistic Computing: WearComp as a New Framework
and Application for Intelligent Signal Processing", Proc. of IEEE, vol.
86, no. 11, 1998.
78. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 78
US Army's Land Warrior
Helmet with OLED
display to show
map & troop
locations
http://en.wikipedia.org/wiki/Land_Warrior
79. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 79
Demo of ISLab@FJU
The X-Eye
80. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 80
The X-Eye
Live photo management
with more comfortable display Demo
81. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 81
X-Eye Components
觸控面板
顯示器 移動電源
Camera
自製
USB
外殼 USB 筆電
連接線
Hub
BeagleBoard
微投影機 SD卡 USB-WIFI卡
鍵盤
USB-RS232
讀卡機
控制線 滑鼠
82. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 82
1st Generation Prototype
83. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 83
Embedded Computer
USER RESET
OMAP3530 Processor
Peripheral I/O •600MHz Cortex-A8
•NEON+VFPv3
•USB Host •16KB/16KB L1
•256KB L2
•JTAG •430MHz C64x+ DSP
•32K/32K L1
•DVI-D video out •48K L1D
•32K L2
•S-Video out •Power VR SGX GPU
•64K on-chip RAM
•SD/MMC+ POP Memory
•256MB LPDDR RAM
•Stereo in/out •256MB NAND flash
•RS-232 serial1
•Alternate power
•USB 2.0 HS OTG
7.6 cm
83
84. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 84
Output - Pico Projector
Very small projector
Large screen for mobile usage
Screen: 15”~30”
Resolution: 480 x 320
Brightness: 7 lumens
Contrast ratio: 1000:1
2009~2018年微型投影機市場預估 (單位:百萬台)
(來源:DisplaySearch Pocket Projector Technology and Market Forecast Report)
85. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 85
Input - Camera
86. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 86
User Interface
Gesture Recognition with
Bare Hand
No keyboard & mouse
Capture egocentric images
87. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 87
Photo Mode
Capture Command: capture images
Switch Command: Mode switch
88. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 88
Manage Mode
Original Photos
Next Command
Previous Command
Switch Command 2(to photo)
88
89. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 89
X-Eye Hardware Architecture
2010.04.25
89
90. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 90
X-Eye Software Architecture
91. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 91
X-Eye Software Stack
Module Version Function
Gesture Command
1.0 Gesture Recognition
Module
OpenCV 1.0 Image Processing
FFMpeg 0.5.1 Audio/Video Codec
QT 4.6.2 Windows Interface
VMWare 6.5.3 Virtual OS
Ubuntu (host) 9.04 Development OS
Ubuntu (client) 9.10 Filesystem
Kernel (client) 2.6.29 Linux Kernel
91
92. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 92
Algorithm (1/2)
Gaussian Mixture Model (GMM)
Model four colors
1
N
1 ( − ( x − mi )T Covi−1 ( x − mi ))
p ( x | c) = ∑ ωi e 2
i =1 2π || Covi ||1/2
Expectation Maximization (EM)
Parameter estimation of GMM
E Step M Step
N
1
∑ E(z
N
1
ω p( x j | m , C ) ωit +1 = t +1
) m = ∑ E(z
t t t
)x j
E ( zij ) = Nω t +1
i i i ij i ij
i N j =1 i j =1
∑ ω tp p( x j | mtp , C tp )
=C t +1 1
t +1 ∑
N
E ( zij )[( x j − mit +1 )( x j − mit +1 )T ]
p =1
N ωi
i
j =1
92
93. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 93
Algorithm (2/2)
Color Identification
c
ˆ =
arg max P( x | c j ), j 1 ~ k
cj
Performance optimization by Look
Up Table (LUT) for real-time
Gesture Recognition
Four gestures: capture, switch,
next, previous,
93
94. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 94
Smart Camera
Egocentric vision needs
small and smart cameras
FJUCam1
FJUCam2
Image Image Image
Capturing Processing Recognition
95. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 95
FJUCam1 - Hardware
• Weight: 35gm
• Power sources: •Size:
• 5V DC current 6 x 4.5 x 5 (cm)
• 5V Cell Battery (W x H x D)
• Power
consumption:
1W
Three Modules
1. Main board, 2. Lens module
3. Storage module
96. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 96
FJUCam1 - Software
Development environment
C Language
PC Windows + Cygwin + GCC
cc3 library (open source developed
by CMU)
97. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 97
FJUCam1 - Face Detection
The Adaboost algorithm
Proposed by Viola and Jones in 2001
Cascaded weak classifiers(21 cascades)
Algorithm refinement
Reduced to 5 cascades
Fixed-point arithmetic
Stream processing for only 64KB
memory utilization
Image
FJUCam Display
Face Detection
99. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 99
FJUCam2 (2/5)
TI OMAP3530@ Cortex-A8 600MHz
@ DSP C64x+ 412MHz
@ PowerVR SGX 530
Integrated L1 memory for ARM (16kB I-
Cache, 16kB D-Cache, 256kB L2)
256MByte low power mobile DDR
512MByte NAND Flash
Mini HDMI interface
Serial port(UART/RS232) x 2
PCB dimensions
USB port (Host) 55mm × 55mm
USB port (Client) Outside dimensions
Micro SD card slot (L×W×H)
USB power Back side – Expansion 63mm 59mm×13 mm
GPIO socket for extra peripherals
Weight : 20 grams
100. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 100
FJUCam2 (3/5)
Software (on board)
Angstrom with Linux
kernel 2.6.32 Camera Input using ARM
VCam Image/Video OpenCV+FFMpeg
DSP
processing demo Image/Video
Using DSP and the Processing
integrated vision
libraries GUI Output using QT
Other supported
embedded OS: easy interface &
powerful computing
Android
We are compiling “VCam Laboratory Manual”
which will be made publicly at the end of 2010
101. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 101
FJUCam2 (4/5)
Algorithms going to be
developed for FJUCam2
Color tracking
Gesture recognition
Face tracking and recognition
Human/hand/finger detection
Event detection
Video summarization
Distributed vision processing
102. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 102
FJUCam2 (5/5)
Possible applications
Wearable computing
Augmented reality
Robotic vision
Visual surveillance
Medical applications for
patient monitoring
103. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 103
4. Conclusions
Image processing
and computer vision
is funny, is cool
Has been realistic,
comes into real life
It is time to study it
Hardware is small, cheap, and
wearable
More robust algorithms
104. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 104
How to Study
Basic courses
Programming skills
Engineering Mathematics
Signal and Systems
Digital Signal Processing
Advanced courses
Digital Image Processing
Computer Vision
Pattern Recognition
Artificial Intelligence
105. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 105
Activities
中華民國影像處理與圖形識別學會(IPPR)
電腦視覺、圖學暨影像處理研討會(CVGIP)
IEEE (國際電機電子學會)
Transactions on
Image Processing
Pattern Analysis and Machine
Intelligence
Medical Imaging
Conferences
ICIP, ICME, ICPR, ...
106. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p. 106
The End
Free for Any Questions
107. Wang, Yuan-Kai(王元凱) Video Surveillance and Sousveillance p.
本簡報授權聲明
此簡報內容採用 Creative Commons 「姓名標示 - 非商
業性台灣 3.0 版」授權條款
歡迎非商業目的的重製、散布或修改本簡報的內容,但
請標明: (1)原作者姓名:王元凱; (2)圖標示:
簡報中所取用的部份圖形創作乃截取自網際網路,僅供
演講者於自由軟體推廣演講時主張合理使用,請讀者不
得對其再行取用,除非您本身自忖亦符合主張合理使用
之情狀,且自負相關法律責任。