If you think there's been a lot of talk about Augmented Reality and Virtual Reality this year, 2018 is going to blow you away. Apple with ARkit, Google with ARCore , Microsoft with HoloLens, Facebook with Oculus and many others are working to transform our Reality with new products and services in the not-too-distant future. Therefore Apple, Microsoft, Google and Facebook is approaching AR/VR from different perspectives and in this session we will try to understand how these different technologies work and which best suits the different areas (industry 4.0, tourism, healthcare, ...)

1. How Augment your Reality:
Different perspective on the Reality / Virtuality Continuum
Matteo Valoriani
CODEMOTION MILAN - SPECIAL EDITION
10 – 11 NOVEMBER 2017

2. How Augment your Reality:
Different perspective on the Reality / Virtuality Continuum

3. Nice to Meet You
matteo.valoriani@fifthingenium.com
Matteo Valoriani, PhD
CEO of FifthIngenium
mvaloriani at gmail.com
@MatteoValoriani
Slideshare: www.slideshare.net/MatteoValoriani
Linkedin: https://it.linkedin.com/in/matteovaloriani
Blog: http://fifthingenium.com/blog
GitHub: https://github.com/mvaloriani

5. How we are arrived here
Virtual Continuum
VR
• Outside-in vs Inside-out
AR
• Visual Inertial Odometry (VIO)
• ARKit/ARCore
• HoloLens
Agenda

6. 2010 - 2017

8. 1968, Apollo Guidance
Computer
2000 Transistor
4k Memory

9. CPU evolution (Transistors)
5M 160M 1B
7B / 2B on
mobile
10B / 3.3B on
mobile

10. 230  4.400  32.000  90.000
(1996) (2003) (2008) (2016)

11. https://www.youtube.com/watch?time_continue=257&v=z6
wvFg-c0Rs

13. RECAP

24. Virtual Reality

25. CAVE vs HMD (Head Mounted Display)

26. Mobile VR

27. Console VR

28. Desktop VR (Outside-in)

29. Pros
• Accuracy (you can add
more trackers around
the room to increase
accuracy)
• Latency very low (reduce
VR sickness).
Cons
• Setup
• Occlusion
• Limitation of the
cameras (field of view)

30. Desktop VR (Inside-out)

31. • .
Pros
• Simplicity / No Setup
• Less hardware needed (no
external camera)
• No limit of space
(potentially)
Cons
• Reduced accuracy and
latency
• Only the object in front to
the user can be tracked
• Powerful hardware

32. VR addons

33. VR Scenarios and Limitations

35. Augmented Reality

37. Optical vs Video see - through

38. Visual Inertial Odometry (VIO)

39. Visual Inertial Odometry (VIO)
6dof in real time :
• 3D of xyz motion (translation)
• 3D of pitch/yaw/roll (rotation)
Pose is tracked:
• via the Camera system, by matching a point in the real world to a pixel on the camera sensor each frame.
• by the Inertial system (accelerometer & gyroscope — Inertial Measurement Unit or IMU)
The output of both of those systems are then combined via a Kalman Filter which determines which of the two
systems is providing the best estimate of your “real” position.

40. b
𝑑 = 𝑥𝑙 − 𝑥 𝑟
𝑏+𝑥 𝑙 − 𝑥 𝑟
𝑍−𝑓
=
𝑏
𝑍
Z =
𝑏∗𝑓
𝑑
Depth Sensing (Multi camera)

41. Single Camera?

42. The secret here is to have *really* good IMU error removal: Dead-reckoning

44. • .
TrueDepth Camera
Face tracking.
Scene Understanding
Detection of horizontal planes (tables, floors,
…) and tracking of object.
Lighting Estimation
Estimation of light available in a scene in
order to apply the correct amount of lighting
to virtual objects.
Hardware and Rendering
Optimizations
Apple A9, A10, and A11 processors.

47. • .
Motion tracking
Estimates a phone’s relative location based
on internal sensors and video footage
Scene Understanding
Uses the camera to detect flat surfaces
Lighting Estimation
Helps virtual props cast accurate shadows
and otherwise fit in with their surroundings
Anchoring objects

50. • .
ARCore vs Tango
Tango’s dedicated infrared depth-
sensing camera to create detailed 3D
meshes.
ARCore has to estimate scale based
on the camera feed, while Tango
directly measures distance.
Nikhil Chandhok (AR director of
product): “ARCore is for all the apps
that we think that users want”

51. HoloLens

53. What is HoloLens?
HoloLens is the first, fully wireless holographic
computer that redefine personal computing and
empowers people in new ways.

54. How to use

55. DEMO

56. Hololens Hadware Blocks

57. Holographic Processing Unit v 1.0
TSMC-fabricated 28 nm co-processor.
24 Tensilica DSP cores (12 clusters)
65 million logic gates (used 50%)
8 MB of SRAM
1GB DRAM
1 Trillion Operation per second
Sensor aggregator with gesture and environment
processing
200x over software implementation
Low Power (<10 Watts)
12 mm
12mm

58. AR Devices

59. Altri framework: Vuforia

60. • .
I've seen things you
people wouldn't believe.
Attack ships on fire off the
shoulder of Orion.
I watched C-beams glitter in the
dark near the Tannhäuser Gate.
All those moments will be lost in
time, like tears in rain. Time to
die.

61. Developer
Resources
Preview Pricing
Documentation
Client SDKs
Example Code
Join Our Community
Thank you!

68. Developer
Resources
Preview Pricing
Documentation
Client SDKs
Example Code
Join Our Community
Thank you!
How Augment your Reality Matteo Valoriani

69. https://medium.com/super-ventures-blog/how-is-arcore-better-than-arkit-5223e6b3e79d
https://www.theverge.com/2017/8/29/16219696/google-arcore-augmented-reality-platform-announce-release-pixel-
samsung
https://www.youtube.com/watch?v=dNXBvDKRg1M
https://www.marxentlabs.com/deep-dive-apple-arkit-game-changer-augmented-reality/
https://developer.apple.com/arkit/
https://developers.google.com/ar/discover/concepts
https://kotaku.com/how-lara-croft-has-changed-over-the-last-16-years-1505856265
http://www.ign.com/articles/2016/07/11/23-perfectly-timed-pokemon-go-screenshots
https://www.wareable.com/trending/trending-vr-add-ons-are-next-big-thing
https://www.howtogeek.com/246333/oculus-rift-vs.-htc-vive-which-vr-headset-is-right-for-you/
https://www.slideshare.net/AugmentedWorldExpo/hiren-bhinde-qualcomm-ondevice-motion-tracking-for-immersive-vr
references