To translate the gesture performed by the user in a video sequence into meaningful symbols/commands, feature extraction is the first and most crucial step in such systems which measures the detected hand positions and its movement track. We propose an efficient approach based on inter-frame difference (IDF) to handle the hand movement tracking, which is shown to be more robust in the accuracy aspect compared to skin-color based approaches. Computational efficiency is another attractive property that our approach greatly improves the processing frame rate to fulfil the demand of a real-time hand gesture recognition system.

1. Presenters
Purohit Pankaj (W579074)
Salagar Muaaz (W579080)
Kulkarni Pranav
(2010BCS203)
Desale Ritesh (2010BCS211)
Seminar Guide
Mrs. S. S. Solapure

2. Agenda
 Introduction
 Reference Paper and Its
Contents
 Application
 Conclusion
 Questions

3. Agenda
 Introduction





5. Introduction
 Gesture Recognition
 Hand Gesture Recognition

7. Introduction (Contd.)
 What is it?
 How it works?

9. Introduction (Contd.)
 What is its NEED?
 Advantages
Natural Interaction
Builds a Richer Bridge
Remote Interaction
Wonderful Gaming Experience

10. Agenda

 Reference Paper and Its
Contents




11. Reference Paper
Movement Tracking in Real-time
Hand Gesture Recognition
Authored by
Hong-Min Zhu & Chi-Man Pun
Department of Computer and Information Science
University of Macau, Macau SAR, China
{ma86560, cmpun} [at] umac.mo
9th IEEE/ACIS International Conference on Computer
and Information Science

12. Reference Paper
 What does it say?
This paper deals with overcoming
of SCHD technique for Hand
Gesture Recognition using newly
improved Algorithm, IFDHD

13. Procedures in General
Framework of Gesture
Recognition

14. Previous Work Done
 Temporal Hand Gesture
 Assumptions
Camera User Synchronization
Uniform Lightening Condition
Simple Background Features
Frame Rate – Gesture Speed
Coordination

15. S C H D
 Skin Color based Hand Detection
 J. Kovac and P. Peer – Designed Skin
Classifier
 Rules
 Pixel is classified as a skin pixel if:
Value of Red > 95, Green > 40 and Blue > 20
&
max{R, G, B} - min{R, G, B} > 15 &
|R - G| > 15 and R > G and R > B

16. Proposed Solution
 Problems with SCHD
Computationally Expensive
Skin-like Object Ambiguity
Illumination Parameters
Skin Color Variation
 Solution – Motivated from BSHD
IFDHD
Inter-Frame Difference based Hand Detection

17. Proposed Solution
Hand
Detectio
n
Module
Motion
Tracking
Module

18. Hand Detection Module
Figure 3.1 Zoomed Mode for Hand Detection Module

19. Algorithm for Hand Detection
Input: Frames Fi = 1..N from video segment
Steps:
1. Convert frame F1 to grayscale
2. Repeat (until end of video segment)
1. Convert frame Fi to grayscale
2. Intensity difference image D0 = |Fi – F1|
3. Binary image I = (D0 > T0)
4. Do image opening on I followed by closing
5. Splitting of Large regions into max size boundary
box
as 60x80
6. Calculate center co-ordinate

20. Experimental Results
 SCHD based Hand Detection






21. Fig. 4. Result of SCHD
(a) original frame

22. Fig. 4. Result of SCHD
(b) Skin Pixel Classification

23. Fig. 4. Result of SCHD
(c) De-noise & Region Connection

24. Fig. 4. Result of SCHD
(d) Region Splitting

25. Fig. 4. Result of SCHD
(e) Centers of Each Region

26. Experimental Results

 Lightening Condition





27. Fig. 5. Effect of Lightening
Condition
(a) Original Frame

28. Fig. 5. Effect of Lightening
Condition
(b) Skin Pixel Classification

29. Experimental Results


 IFDHD based Hand Detection




30. Fig. 6. Result of
IFDHD
(a) 1st Frame

31. Fig. 6. Result of
IFDHD
(b) 11th Frame

32. Fig. 6. Result of
IFDHD
(c) Subtraction: (b) -
(a)

33. Fig. 6. Result of
IFDHD
(d) Threshold of (c)

34. Fig. 6. Result of
IFDHD
(e) De-noise

35. Fig. 6. Result of
IFDHD
(f) Region Splitting

36. Fig. 6. Result of
IFDHD
(g) Centers of
Regions

37. System Domain (Contd.)

38. Movement Tracking Module
Figure 3.2 Zoomed Mode for Movement Tracking Module

39. Algorithm for Movement
Tracking
Input: Region centers Detected in each frame
Steps:
1. Initialize the start of frame
2. Repeat (for each frame > 1)
1. Identify tail locations and store
2. Calculate matrix of distances between centers and tail
locations
3. Repeatedly select – min(Distance ( I ), Distance ( J ))
4. If Distance( I ) < Threshold then append Center to Gesture
and delete Distance( I ) Else initialize new Gesture start
location
5. Select Gesture Frame that has the maximal standard
deviation

40. Experimental Results



 SCHD based Movement Tracking



41. Fig. 7. SCHD Based Movement Tracking
First Row: Last Frame of Video Segment

42. Fig. 7. SCHD Based Movement Tracking
Second Row: Detected Digit Track

43. Fig. 7. SCHD Based Movement Tracking
Third Row: Smoothed Track

44. Experimental Results




 IFDHD based Movement
Tracking


45. Fig. 8. IFDHD Based Movement Tracking
First Row: Last Row of Video Segment

46. Fig. 8. IFDHD Based Movement Tracking
Second Row: Detected Digit Track

47. Fig. 8. IFDHD Based Movement Tracking
Third Row: Smoothed Track

48. Experimental Results





 Efficiency Measurement

49. Efficiency Measurement
Table 1. Comparing the Efficiencies of SCHD and IF

50. Outline


 Application Scenario



51. Reference Paper
American Sign Language Recognition
System for Hearing Impaired People Using
Cartesian Genetic Programming
Authored By
Fahad Ullah
Department of Computer Systems Engineering,
University Of Engineering & Technology,
Peshawar, Pakistan
Proceeding of 5th International Conference on
Automation, Robotics and Applications, New
Zealand

52. Application Scenario
 Why the interfaces are changing ?
 How many Apps Out there?
 Have you tried X-box,PSP-2,Mac-OSX
 January 9, 2012, 66 million Xbox 360 consoles
have been sold worldwide.
 New era of Interfaces

53. Application Scenario
 What if you can’t speak?
 ASL
 CGP (Cartesian Genetic Programming)
 How it works?
 Genetic programming an Overview:
 Probabilistic search
 Darwinian principle of natural selection
 Naturally occurring genetic operations such
as crossover and mutation.

54. • Better individuals are preferred
• Best is not always picked
• Worst is not necessarily excluded
• Nothing is guaranteed
• Mixture of greedy exploitation and
adventurous exploration
• Similarities to simulated annealing (SA)
Probabilistic Selection Based
On Fitness

55. Workflow

57. ASL using CGP
26 English language alphabets are trained
and Identified
The system uses 26 binary images
representing the different alphabets
Mentioned system with a Dictionary
correction ability in order to increase the
overall accuracy of the system.

58. Outline



 Conclusion


59. Conclusion
 Proposed IFDHD
 Serving Feature Extraction Stage
 Overcoming the pitfalls of SCHD

60. Outline




 Questions

61. Questions, IF ANY?
Q?
Thank You
