Devices like HoloLens and Tango provide powerful spatial awareness capabilities that go beyond simple AR to enable truly Mixed Reality applications. Leveraging these capabilities requires an understanding of these technologies as well as new ways of thinking about user interface design and gameplay. Aaron shares his team's experiences developing apps for spatially aware devices, including Behemoth which won Best Spatial Mapping at the SFVR HoloLens hackathon and Ghostly Mansion which won Best Entertainment/Game app in Google's Project Tango development contest. He will discuss how these devices differ, how he approaches design to leverage their unique features, and lessons learned along the way.

4. Mixed Reality

5. Spatial Awareness
Motion Tracking – Depth Sensing – Area Learning
Surface Reconstruction – Plane & Void Detection

6. Spatially Aware Devices

7. Now available in Australia, Canada, France, Germany, Ireland, New Zealand, United
Kingdom, and United States.

8. Wearable Windows 10 PC

9. Infrared Camera Spatial Mapping

10. Motion and Gesture Tracking

11. HoloLens Development Environment
Microsoft / HoloToolKit / SpatialMapping SpatialProcessing

12. Best Spatial
Mapping
WINNER
SFVR HoloLens
Hackathon 2016

13. Behemoth for HoloLens
• Enabling epic cinematic
experiences anywhere!
• Discover mysterious dangers
lurking beyond your walls
• Explore space for resources

16. Lenovo Phab2 ProDevelopment Kit

17. Tango Development Environment
 SDK Options
• C, Java, Unity
 UX Framework
 Active Community

18. Best
Entertainment
WINNER
Project Tango App
Contest 2015

19. Story-Driven Physical Hidden-Object Game

20. Motion Tracking driven design
“In this mystery sleuthing title, you play an
imprisoned spirit, trying to find peace by
solving the mystery of your own death.”
“It takes place in a spooky 3D mansion, and
the entire game is controlled by walking
around holding Tango.”

21. Mixed Reality with Tango
Google I/O 2016

22. Voxelized Environment Retheming
Cyrus Lum
“Worlds Away”
FINALIST
Lenovo Tango
Hackathon
2016

23. Mixed Reality UX
• Embrace 3D Space
• Prioritize Motion UI
• 3D Audio is Critical
• ReFlow to environment

24. Mixed Reality ReFlow
Each environment is
unique
• AI/CSS for MxR
• Mesh generation
• Plane extraction
• Void identification

26. R-7 Smartglasses

27. MxR Dev Kit

28. Apple iPhone 7 Plus and Beyond
• Dual-cameras
• PrimeSense
• Metaio

29. Project
Alloy

30. Santa Cruz Prototype

32. 2017?