1. VIRTUAL PIANO
TEAM: THE DAYDREAMERS
SAHIL CHELARAMANI (20162051)
PRANAV DHAKRAS (20162303)

3. DETECTING KEYS
 Get calibration template
 Otsu’s thresholding for multiple thresholds

4. DETECTING KEYS
 Connected component analysis
 Unique ID for each key

5. USING HOMOGRAPHY
 Extract feature points from template and video
frame
 Match points
 Find homography
 Issues due to symmetry

6. CONTOUR FITTING FOR SKIN COLOUR
 Apply thresholding to get mask of hands
 Very sensitive to skin colour and factors such as
nail paint

7. CONTOUR FITTING FOR SKIN COLOUR
 Get convex hull of hands
 Find highest y-point from hull
 Approximate it as fingertip
 Problems:
 Works for only one finger
 Problem with hovering fingers

8. DETECTING WHITE KEYS
 Sobel operator
 Crescent shaped response when key is pressed

9. ACTIVITY MAP AND SMART THRESHOLDING

10. DETECTING BLACK KEYS

11. FILTERING DUPLICATE KEY PRESSES
 Problems due to activity map
 Use of queue to solve the issues

12. RESULTS

13. RESULTS

14. FAILURE CASES

15. FUTURE WORK
 Skin detection
 Using CNNs to detect presses
 Depth maps

16. THANK YOU !