BayCHI June 14, 2005, program: Why is video conferencing not ubiquitous? Over 75 years after Bell Labs released the first videoconferencing system on April 7, 1927, we still do not experience videoconferencing as a common means to communicate today. Milton attributes the lack of adoption to limitations in technology and inadequate understanding of visual communication science.

1. Experiments in Videoconferencing Milton Chen CTO http://vseelab.com

2. The VSee Auditorium desktop interface 15’ x 5’ video wall

5. What if there is no network infrastructure ?

6. Office of Secretary of Defense, State Department, NATO, United Nations … Strong Angel Kona, Hawaii 17-22 July 2004 VSee was selected as the real-time communication system

7. VSee at Strong Angel Provide global communication from a temporary shelter VIP presentation between Kona and DC

8. Ad-hoc peer-to-peer WiFi ~ 0.5 mile ~1 - 10 mile

9. Experiment 1: convoy protection VSee hops from car to car

10. Experiment 2: air-to-surface Can also airdrop arbitrary data setup screen shot

11. Experiment 3: ocean search and rescue

12. Experiment 3: ocean search and rescue The bottom video was from the live underwater camera held by the swimmer. The map with GPS annotation was shared using VSee setup screen shot

13. Experiment 3: ocean search and rescue

16. March 2005 From VSee deployment team

17. VSee for tsunami relief UN headquarters in Jakarta

18. VSee in Darfur for refugee management CARE International field office Sudan, Africa

19. but

21. VSee Peer-to-peer wireless How well can we judge eye contact “ The heart is stirred more slowly by the ear than by the eye .” – Horace

22. Eye contact stirs us to action [Sharbat Gula, photographed by McCurry ‘83]

23. Eye contact fires up our brain [Kampe et al. ’01 Nature ]

26. Eye contact is difficult Looking into the camera Attempting eye contact

27. Solutions to eye contact Half-silvered mirror [Rosenthal ’47] MAJIC [Okada, et al. ’94] ClearBoard [Ishii, et al. ’92] GazeMaster [Gemmell, et al. ’00]

28. Methodology Observers watch videos of looker Large display with camera at the center

29. Eye contact?

30. Sensitivity is asymmetric * 16 observers judged recorded videos of 1 looker

31. An anatomical explanation looking at you looking sideways looking up looking down eye closing Illustrations from The Artist’s Guide to Facial Expression [Faigin ’90]

32. VSee Eye contact How well can we judge lip sync “ We shape our tools, and there after our tools shape us” - Marshal Mcluhan

35. Conventional lip synchronization encode network decode A a v time Unsynchronized encode network decode sync a , v Audio delay line A delay skew

37. A new lip sync method encode network decode sync synchronized and low perceived latency a v a v encode network decode A a v time Unsynchronized encode network decode sync a , v Audio delay line A Round trip delay

39. Perception of variable AV skew * 16 subjects judging recorded videos

40. VSee Eye contact Lip sync What frame rate is necessary “ We express ourselves into existence .” – Iris Murdoch

42. Gesture Detection Algorithm input image frame difference after erosion Visualization of algorithm

43. Gesture sensitive transmission allows dynamic discussion 15 fps ~0.2 fps 0.2 fps * 8 groups of 4 people during a discussion * requires 10% of full motion bandwidth

44. Other studies [ Conveying Conversational Cues Through Video PhD Dissertation, 2003] When is a smile not a smile Value of f2f for discussion Visualizing the pulse of Classroom

45. VSee Eye contact Lip sync Gesture Telework “ Laugher is the shortest distance between two people ” – Victor Borge

46. VSee customers

50. I love to hear from you [email_address] Summary VSee Eye contact Lip sync Gesture Telework