HoloLens Development: the Next Steps #globalazure
•Descargar como PPTX, PDF•
3 recomendaciones•1,475 vistas
Want to learn how to start developing a mixed reality experience? This talk will introduce new concepts and build on existing ones for doing just that. Learn how to take advantage of gestures, how to leverage voice commands, learn to master the developer tools such as the emulator and perception simulation, and much more. Get to grips with the development cycle and get into the details of the HoloLens development experience. At the end of this talk you will have the tools and knowledge to build more immersive and natural holographic experiences that take full advantage of mixed reality and isn’t merely an augmented or virtual experience. We will also explore ways to take it further to build a more immersive, more natural, more believable?
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a HoloLens Development: the Next Steps #globalazure
Similar a HoloLens Development: the Next Steps #globalazure (20)
Más de Kanwal Khipple
Más de Kanwal Khipple (20)
Último
Último (20)
HoloLens Development: the Next Steps #globalazure
- 1. HoloLens Development: Next Steps Learn how to start developing a mixed reality experience and be inspired on how you can leverage Azure By: Kanwal Khipple & Mike Morawski #globalazure
- 2. SPEAKER | AUTHOR | CEO KANWAL “SUPER GOOD LOOKING” KHIPPLE @kkhipple bit.ly/linkedinkk 2014 2015
- 3. DEVELOPER MIKE “BUILDER AT NIGHT” MORAWSKI http://bit.ly/2ofNkKj
- 4. Developing next generation experiences WHY AUGMENTED REALITY DEVELOPMENT BASICS AZURE INTEGRATION
- 5. Developing next generation experiences WHY AUGMENTED REALITY DEVELOPMENT BASICS AZURE INTEGRATION
- 6. Unveils latest technology to transform the elevator industry THYSSENKRUPP
- 7. Retail store experience that defines how we shop in the future
- 8. What we will be showcasing today SHOPPING EXPERIENCE
- 9. Developing next generation experiences WHY AUGMENTED REALITY DEVELOPMENT BASICS AZURE INTEGRATION
- 10. Powerful in a small package HOLOLENS
- 11. Ability to mix virtual objects into the real world FRAGMENTS
- 12. Tutorials that will help you start development from scratch https://developer.microsoft.com/en-us/windows/holographic/academy HOLOGRAPHIC ACADEMY
- 13. Step-by-step walkthrough on how to install the development kit https://developer.microsoft.com/en-us/windows/holographic/install_the_tools INSTALLATION CHECKLIST
- 14. Intended to accelerate development of Windows holographic applications https://github.com/Microsoft/HoloToolkit-Unity HOLOTOOLKIT
- 15. Latest gibhub release compiled into a unity file http://holotoolkit.azurewebsites.net/ HOLOTOOLKIT UNITY PACKAGE
- 16. Make sure there’s a lot of light when you are developing; however, development can be done all on an emulator using emulated rooms BRIGHT ROOMS
- 17. Accelerate HoloLens development HOLOTOOLKIT FEATURES
- 18. Leverage look at an object you intend to interact with GAZE
- 19. Two Basic Movements (Air Tap & Bloom) with variations GESTURES
- 20. Speak directly to Cortana VOICE & SOUND
- 21. Great applications ask questions SPEECH DESIGN • What actions can be taken through speech? • Is speech input a good option for completing a task? • How does a user know when speech input is available? • Is the app always listening? • What phrases initiate an action or behavior? • What is the interaction dialog between app and user? • Is network connectivity required?
- 22. Map objects to the real world SPATIAL MAPPING
- 23. Everyone gets a HoloLens demo! WHO WANTS ONE?
- 24. Developing next generation experiences WHY AUGMENTED REALITY DEVELOPMENT BASICS AZURE INTEGRATION
- 25. IOT and integration of Sensor Fusion-like data streams will exponentially increase the growth of data GROWTH OF DATA
- 26. Ability to log millions of events per second in near real time AZURE EVENT HUB
- 27. Store and analyze petabytes of data AZURE DATA LAKE
- 28. Build prediction models and improve the retail experience AZURE-BASED MACHINE LEARNING
- 29. You can find this slide on http://slideshare.net/kkhipple THANK YOU!
Notas del editor
- Want to learn how to start developing a mixed reality experience? This talk will introduce new concepts and build on existing ones for doing just that. Learn how to take advantage of gestures, how to leverage voice commands, learn to master the developer tools such as the emulator and perception simulation, and much more. Get to grips with the development cycle and get into the details of the HoloLens development experience. At the end of this talk you will have the tools and knowledge to build more immersive and natural holographic experiences that take full advantage of mixed reality and isn’t merely an augmented or virtual experience. We will also explore ways to take it further to build a more immersive, more natural, more believable?
- SOURCE: https://www.thyssenkrupp.com/en/newsroom/press-releases/press-release-114208.html
- 2gb ram 64gb hdd Atom processor Custom holographic processing unit HPU processes a large amount of data per second from the sensors. Microsoft HoloLens understands gestures and where you look, and maps the world around you, all in real time. Trillion calculations per second Sensor fusion Advanced sensors capture information about what you're doing and the environment you’re in. Microsoft HoloLens can see, map, and understand the physical places, spaces, and things around you Microsoft HoloLens 1.0 specs CategorySpecsDisplaySee-through holographic lenses (waveguides) 2x HD 16:9 light engines Automatic pupillary distance calibration 2.3M total light points holographic resolution, 2.5k light points per radianSensorsInertial Measurement Unit, 4x environment understanding cameras, mixed reality capture, 4x microphones, ambient light sensorProcessorCustom Microsoft Holographic Processing Unit HPU 1.0, Intel 32-bit architectureRAM2GBStorage64GBWeight579g (1.2lbs)Camera2MP photos, HD videoAudioExternal speakers, 3.5mm audio jackConnectivityWi-Fi 802.11ac, Bluetooth 4.1 LE, Micro-USB 2.0Power2-3 hour active use battery life, 2 weeks standby, passive coolingOSWindows 10 with Windows Store Human Understanding: spatial sound, gaze tracking, gesture input, voice support
- Project setup Spatial Sharing - You can connect up to 16 devices. Shared experiences. Scripts
- Where your head is pointing (not your eyes). Where you are looking Primary form of input for mixed reality Used in our example to capture what is looked at for future analytics on Azure In real world typically look at an object you intend to interact with
- Tap is the most frequent Bloom is like an exit Two Basic Movements Select (aka Air Tap) Home (aka Bloom) Variations Tap - A Select press and release. Hold - Holding a Select press beyond the system's Hold threshold. Manipulation - A Select press, followed by absolute movement of your hand through 3-dimensional world. (I have not used or seen this before) Navigation - A Select press, followed by relative movement of your hand or the controller within a 3-dimensional unit cube (I have not used or seen this before)
- Voice recognition system is very accurate and fast. Emulator is a bit slow but a physical device is really quick Does work offline "Hey cortana, what can I say" to show basic commands (Can’t be used in app) Saying Select can be used instead of air tap SOUND DESIGN Atmosphere sound and
- Four cameras Work kinda like a Kinect. Mapping the room in 3D Kinda accurate. Polygon count. Not 100% live. Range is 5m. Not entire environment. You have to walk-in
- HoloToolKit GitHub How the scene is setup HoloCollection HUDs Managers – global services Script libraries (input, sharing, sound, utilities) #Gaze1 Gaze is typically used in a ray cast. Very simply, a ray (think of a laser) is drawn out and the first thing it touches can generate a point at which something can be done, or the target it hit can be modified (i.e. hologram is hit, so highlight it) Gets the position of your head Draws line using direction head is facing (not eyes) Can filter objects from what is hit Can do intersects with holograms and real world mesh (spatial map) i.e. finding a place to put something in a real room (wherever laser hits can be referenced on the object it hit for location). Holograms placed in the air/ground also act a similar way. Code demo: TapToPlace.cs Update routine #realtime Explain how checkoutBox gets highlighted by physics raycast OnInputClicked Notify the store manager that an item was placed in the box Physics & dropping item into box Gesture #GAZE2 #GESTURE1 I don’t use gestures directly but we could perhaps explain what’s available and I could show how it may be implemented. Code demo: TapToPlace.cs OnINputClicked #gesture1 gesturesinput #gesture2 Monobehaviour IInputClickHandler #gesture3 Voice (Commands) Voice recognition system is very accurate and fast. Emulator is a bit slow but a physical device is really quick Does work offline "Hey cortana, what can I say" to show basic commands (Can’t be used in app) Saying Select can be used instead of air tap DEMO Managers->SpeechManager (explain phrases /keyword setup and managers) Holocollection (explain receivers of events) Speech listener custom code (reset world/show direction) Spatial Mapping This is the window to mixed reality Items can be 'anchored' to a location in a room (it remembers where in your house something ins). Items can also be anchored to the hololens (i.e. fixed 1 meter ahead) Allows for physics against the real world Can be used for navigation (i.e. a game character moving around it) Allows for occluding holograms behind real world items Code demo: TaptoPlace.cs OnInputClicked DrawVisualMeshes - turns on mesh TaptoPlace.cs Update Tap to place and showing Update function where item is placed against world map Spatial Understanding (Extension of spatial mapping) Understand that real world understanding is limited. I.e. the spatial map is one big jumble of polygons Some understanding can often be made of basic somewhat foundational items: Ceilings Floors Doors Walls Couches/Chairs (lesser extent?) Understanding works by seeing if there is a large portion of a mesh being flat for example (if its above us, and the polygon faces are pointing downward) it can be somewhat assumed that it is a ceiling. HOLOLENS DEMO Demo of the solution. Mike wearing Hololens. Connected to your computer. Ppl will be able to see what you are seeing on the project. AZURE EVENT HUB Code demo: StoreManager.cs - Event Hub submits to Event Hub using Telemetry plugin Can demonstrate simple data coming through using Server Bus explorer (below is about all that would be shown – not sure if this needs to be shown streaming or just reflected upon) GAZE GESTURE VOICE SPATIAL MAPPING
- Event Hubs (just $11/month to capture 1 MB/sec of events sent to your Event Hub) Ability to log millions of events per second in near real time Elastic scaling support with the ability to scale-up/down with no interruption Support for multiple protocols including support for HTTP and AMQP based events Flexible authorization and throttling device policies Time-based event buffering with event order preservation Azure EventHub > Azure Stream Analytics > Azure Storage Blobs > Azure Data Factory (ADF) > Azure Data Lake Store
- INJEST Store and analyze petabyte-size files and trillions of objects STORE Affordable and cost effective (Azure Data Lake Storage) ANALYSIS trillions of files where a single file can be greater than a petabyte in size which is 200x larger than other cloud stores (HDInsight, Azure Data Lake Analytics)
- Leverage Azure machine learning to grab the data from the Event Hub, run some logistic regressions, or other predictive models to get real results CUSTOMER EXAMPLE - at a large soft-drink bottling company we were able to confirm that temperature is the most important influencer of demand for their beverage products AND exactly how many units of beverage product will be sold for each degree of increase/decrease in temperature. identify the impact of other variables like holiday stock-ups, sporting events and unemployment This allows them to better plan for promotions and price changes and optimize delivery routes CUSTOMER EXAMPLE – for a website, we leveraged A/B testing is jargon for a randomized experiment with two variants, A and B, which are the control and treatment in the controlled experiment. It is a form of statistical hypothesis testing with two variants leading to the technical term, Two-sample hypothesis testing, used in the field of statistics. Retailers could run A/B Testing to determine the best Store or Shelf Layout. Consumer Goods companies could run A/B Testing to determine product packaging or shelf layout. Solutions like the one can be easily used for A/B Analysis to determine the most optimal shelf layouts store layouts and product packaging.