Brief introduction to mixed reality including original example projects in virtual reality (VR), augmented reality (AR), augmented virtuality (AV), and real virtuality (RV), AKA ubiquitious and mobile computing. Includes examples of whole-body immersion, gesture-based interaction, and on-body interaction. Also, explore the state of the art in the scientific literature and the market place, including Oculus Rift, Microsoft Hololens, Google Cardboard, and Google Tango.

1. Mixed-Reality Technology and Whole-Body Interaction for Active Lifestyles
Mario Romero
2015/12/03
Photo: Fredrik Lilkaer. Project: SolarSense (AGI2013)

2. A note:
• All of the images in this presentation have
hyperlinks to original content.
2015/12/03 Romero 2

3. Outline
• What is Mixed Reality?
• A vision of the near future
• Select portfolio
• Discussion: how can tech promote
healthier lifestyles?
2015/12/03 Romero 4

4. Disclaimer
• Not talking about aging
– Talking about active live styles for all ages
• Not talking about all technology
– Talking about mixed-reality
2015/12/03 Romero 5

5. 2015/12/03 Romero 6
Is this you, sitting, now?
Enjoying Real Virtuality,
but sedentary?

6. What is mixed-reality?
It is the merging real and virtual objects and
interfaces to produce new experiences where the
physical and digital co-exist and interact in real-time.
2015/12/03 Romero 7

7. Mixed-Reality Spectrum
2015/12/03 Romero 8

8. 2015/12/03 Romero 9

9. Augmented and Virtual Reality Spectrum
2015/12/03 Romero 10

10. Augmented and Virtual Reality Spectrum
2013
2015/12/03 Romero 11

11. Augmented and Virtual Reality Spectrum
2013
2014
2015/12/03 Romero 12

12. Augmented and Virtual Reality Spectrum
2013
2014
2015
2015/12/03 Romero 13

13. Augmented and Virtual Reality Spectrum
2013
2014
2015
2015/12/03 Romero 14

14. A vision of the future
2015/12/03 Romero 15

15. Skin Input
Harrison 2010
2015/12/03 Romero 16

16. Muscle Input
Saponas 20092015/12/03 Romero 17

17. Bone Output
Walker 2005
2015/12/03 Romero 18

18. Multi-surface Input
Synaptics Fuse 2006
2015/12/03 Romero 19

19. Air periphery phone input
Fuffr 2015
2015/12/03 Romero 20

20. Virtual Reality
Oculus Rift 2016
2015/12/03 Romero 21

21. Mobile VR
2015/12/03 Romero 22

22. Mobile 3D Intelligence
Google Tango 2014
2015/12/03 Romero 23

23. Full Mixed Reality
Microsoft Hololens 2015
2015/12/03 Romero 24

24. Muscle Output
Lopes 2013
2015/12/03 Romero 25

25. Shared Augmented Reality without wearing gear
Magic Leap 2017 (?)
2015/12/03 Romero 26

26. My studio
2015/12/03 Romero 27

27. 2015/12/03 Romero 28

28. • Research
• Visualization Supported Collaborative Work
• Foundational Technology
• User Evaluations
• Showcase and classroom environment
• Outreach
Students’ Resources:
Visualization Studio
2015/12/03 Romero 29

29. • High-resolution projection
wall with stereoscopy
• Oculus Rift
• Cinema quality audio
• High-definition video
communications with eye
contact
• Holographic display
• Multi-touch interactive
surfaces
• Eye tracking
• GPU-based computing cluster
• Diverse interaction and
sensor systems (haptic,
mocap, etc.)
• Haptic Devices
• 3D printer
• Epson Moverio
Technologies in
2015/12/03 Romero 30

30. 2015/12/03 Romero 31

31. VisualizationStudio
2015/12/03 Romero 32

32. Students
2015/12/03 Romero 33

33. Select Portfolio
2015/12/03 Romero 34

34. Virtual Sculpting
2015/12/03 Romero 35

35. 2015/12/03 Romero 36

36. 2015/12/03 Romero 37

37. 2015/12/03 Romero 38

38. Embodied Sculpting
2015/12/03 Romero 39

39. MegaMind Tekniska Museet – Forma i Luften
Hands-on interaction with 3D, haptics, sound
2015/12/03 Romero 40

40. Base Jumper
• Octree rendering
• Computer vision body
control
• Link (not on the image)
2015/12/03 Romero 41

41. DangeRoads
2015/12/03 Romero 42

42. FaceUp
2015/12/03 Romero 43

43. PodRacer
2015/12/03 Romero 44

44. PodRacer
2015/12/03 Romero 45

45. 2Pacs
2015/12/03 Romero 46

46. From the stormtroopers:
"Devoid of colour differentiation
(Stormtroopers see everything
in shades of green) made it hard
to spot the ghosts." A universal
consideration for interaction
design: accessibility and color
blindness.
2Pacs
2015/12/03 Romero 47

47. InvadAR
2015/12/03 Romero 48

48. YA3
2015/12/03 Romero 49

49. 2015/12/03 Romero 50

50. 2015/12/03 Romero 51

51. 2015/12/03 Romero 52

52. 2015/12/03 Romero 53

53. Mad Sand
2015/12/03 Romero 54

54. Teamtris
2015/12/03 Romero 55

55. BrARwl
2015/12/03 Romero 56

56. Padawan 101
2015/12/03 Romero 57

57. Shmooning
2015/12/03 Romero 58

58. Blopper
2015/12/03 Romero 59

59. Resources
• ACM SIGGRAPH
• ACM CHI
• ASSETS: ACM International Conference on
Computers and Accessibility
• CSCW: ACM conference on Computer
Supported Cooperative Work
• DIS: ACM conference on Designing Interactive
Systems
• ECSCW: European Conference on Computer-
Supported Cooperative Work: ACM
conference on supporting group work
• HRI: ACM/IEEE International Conference
on Human–robot interaction
• ICMI: International Conference on
Multimodal Interfaces
• ITS: ACM conference on Interactive Tabletops
and Surfaces
• IUI: International Conference on Intelligent
User Interfaces
• MobileHCI: International Conference on
Human–Computer Interaction with Mobile
Devices and Services
• NIME: International Conference on New
Interfaces for Musical Expression
• Ubicomp: International Conference
on Ubiquitous computing
• UIST: ACM Symposium on User Interface
Software and Technology
• i-USEr: International Conference on User
Science and Engineering
2015/12/03 Romero 60

60. 2015/12/03 Romero 61

61. Thank you!
marior@kth.se
Discussion?
2015/12/03 Romero 62
https://goo.gl/ZkOI7t