ISMAR17 - Augmenting, Mixing & Extending Reality on the web

1. Rob Manson, CEO & co-founder - https://awe.media - follow me on twitter @nambor Augmenting, Mixing & Extending Reality on the web

2. Who am I? Rob Manson, CEO & co-founder https://awe.media

3. Who am I? Rob Manson, CEO & co-founder https://awe.media The original evangelist for Mixed Reality using open Web Standards

4. Who am I? Rob Manson, CEO & co-founder https://awe.media The original evangelist for Mixed Reality using open Web Standards Invited Expert

5. Who am I? Rob Manson, CEO & co-founder https://awe.media The original evangelist for Mixed Reality using open Web Standards Invited Expert W3C

6. Who am I? Rob Manson, CEO & co-founder https://awe.media The original evangelist for Mixed Reality using open Web Standards Invited Expert W3C Khronos Group

7. Who am I? Rob Manson, CEO & co-founder https://awe.media The original evangelist for Mixed Reality using open Web Standards Invited Expert W3C Khronos Group ISO

8. A brief history…

9. How did we get here?

17. How did we get here? 2017 The web gets Computer Vision!

18. What predictions did we get right? Mixed Reality will be possible in the web

19. What predictions did we get right? Mixed Reality will be possible in the web ARStandards Workshop Seoul, 2010

20. What predictions did we get right? Mixed Reality Web Standards test harness

21. What predictions did we get right? Mixed Reality Web Standards test harness isweb3here.com, 2011

22. What predictions did we get right? OS-level spatial tracking would be released

23. What predictions did we get right? OS-level spatial tracking would be released W3C WebVR Workshop, 2016

24. But getting the timing right is really hard…

25. But Tim is only right if you give up too soon!

26. Here are the different ways people can interact with Mixed Reality

27. Mixed Reality Interaction Mode – Public - Whole body in fov - More spectators - Limited distribution

28. Mixed Reality Interaction Mode – Personal - Upper/whole body in fov - Fewer spectators - Wider distribution

29. Mixed Reality Interaction Mode – Private - Extremeties or head in fov - 1-2 direct participants - Wider distribution & locations

30. Mixed Reality Interaction Mode – Intimate - Only extremeties in fov - 1 direct participant - Pervasive but currently limited distribution

31. Where are we up to now?

32. What is Mixed Reality? Image Credit Paul Milgram - 1994

33. What is Augmented Reality? Augmented Reality allows the user to see the real world, with virtual objects superimposed upon or composited with the real world. Ron Azuma - 1997 Augmented Reality… 1. Combines the real and virtual 2. Is interactive in real time 3. Is registered in 3D

34. What is Extended Reality? Image Credit

37. Image Credit

38. How do you “View” VR & AR? Have a suitable HMD v 1 2Have a valid account with the HMD vendor's app store 3 Complete the HMD vendor's first use 4 v Search for & find the app in their app store 5 Download/install the app through their app store 6Often requires system or app updates also v 7 Select/play the downloaded app ! MOBILE ONLY APP Have a valid account with the Mobile OS vendor's app store v 1 2 Find the app in their app store3 Download/install the app through their app store 4 v Often requires system or app updates also 5 Download/install the app through their app store 6 Select/play the downloaded app v 7 ! View promo for app on their website WEB APP 1 Click on a web link to open it in your browser on any device HMD ONLY APP

39. How do you “Create” VR & AR? MOBILE ONLY APPs WEB APPSHMD ONLY APPS Hire or be a developer v 1 2 Download & get to know (& deal with the various issues/quirks) the developer SDK for the target platform 3 Develop your app/s 4 v Test it on the target HMD vendor platform/s 5 Convert media and 3D assets to the app/s format 6 Sign & submit your app/s to the app store/s & wait for approval/s v 7 Publish & distribute links to one or more app stores ! Download & get to know (& deal with the various issues/quirks) the developer SDK for the target mobile OS v 1 2 Develop your app/s 3 Test it on the target HMD vendor platform/s 4 v Convert media and 3D assets to the app/s format 5 Sign & submit your app/s to the app store/s & wait for approval/s 6 Publish & distribute links to one or more app stores v 7 Hire or be a developer 8 Create promotional web content to drive people to the app/s ! 8 Create promotional web content to drive people to the app/s Get an account with awe.media *blatant plug J v 1 2 Upload any types of common media you need (photos, videos, 3D & more) 3 Manipulate these objects and add rich interactions to create your VR & AR experiences 4 Open your browser on any type of device v 5 Share a single link that points directly to your web- based VR/AR experience

40. What’s evolving as we speak?

41. Depth sensing Depth Sensing is starting to become commonplace Image Credit

42. OS-level pose tracking Image Credit

43. WebVR/WebAR Image Credit

44. WebVR/WebXR Image Credit

45. Deep/Machine Learning Image Credit

46. More cameras! Image Credit

47. What’s needed in the future? (The missing pieces)

48. Camera intrinsics web API Image Credit A more accurate underlying geometry for Computer Vision

49. Optimised pixel access Zero copy texture transfer & faster workers Image Credit

50. Lets look at how this is all playing out in the market

51. Why has VR exploded recently? VR has things to sell Image Credit

52. Why has VR exploded recently? The power of the s-curve Image Credit

53. Where are we now? In reality, VR is a tiny tiny market right now

54. Where are we now? In reality, VR is a tiny tiny market right now More iPhone 8s than HMDs by the end of the year!

55. What’s happening in the global mobile market? Image Credit

56. The good news for iOS is the majority are now iOS11 Image Credit

57. Where are we now? Installed web browsers 20 MILLION (END 2017) Includes both the mobile and HMD users above 2 BILLION 3 BILLION Mobile devices HMD only

61. Immersive Computing will become Pervasive Image Credit What is likely in the future?

62. What are the real market trends? This will take more than 3-5 years Image Credit

63. Mainstream adoption won’t “start” until at least 2020 Image Credit How long until we get our magic glasses?

64. What will drive market changes between now & then? Convenient multi-use mobile form factors are the key today Image Credit

65. Questions? / Discussion Rob Manson, CEO & co-founder - https://awe.media - follow me on twitter @nambor