Slides Vikas Batra recently used in his discussion w/ mentees of The Product Mentor.
Synopsis: In this talk Vikas will share recent developments in the field of Virtual Reality(VR) and Augmented Reality (AR) . Share use-cases on how AR is being used by enterprises to help you identify how you could use it to gain competitive advantage in your market.
The Product Mentor is a program designed to pair Product Mentors and Mentees from around the World, across all industries, from start-up to enterprise, guided by the fundamental goals…Better Decisions. Better Products. Better Product People.
Throughout the program, each mentor leads a conversation in an area of their expertise that is live streamed and available to both mentee and the broader product community.
http://TheProductMentor.com
3. The Objective
Augmented Reality and Virtual Reality
Examine
Competitive Advantage ?
Address New Market/Customer Segment ?
Process of Assessment
3
4. Spatial User Interface
New Compute Interface
Redefines human interaction with Physical Objects and Physical Space
1. 3-Dimensional, Computer-generated, Graphical Spatial View
1. augments actual physical view (Augmented Reality), or
2. eliminates actual physical view (Virtual Reality)
2. Hands Free. Controlled by hands gestures, head/body movements
4
22. Software
Recognize and Track Environment
Define AR Experiences
Execute AR Experiences
Integration with Existing Systems
50+ AR Authoring Platforms/SDKs
Comparison at Social Compare
http://socialcompare.com/en/comparison/augmented-reality-sdks
22
24. The Big Picture24
Internet of Things
Artificial Intelligence/
Analytics
Humans
Data
Automated
Control
Data
Manual
Control
‘Pull’
‘ Contextually
Relevant Push’
25. The Promise
Augmented Reality
3-Dimensional, Augmented Views (Audio-Video)
Hands Free Operation - Input/output from hand gestures, voice,
camera, sensors …
Larger, adjustable display area
Greater Comfort,
Always with you. ‘Contextually Relevant Push’
Virtual Reality
Immersive Experiences
Significantly better end-user experience
compared to Smartphone/PC/Tablets
25
26. The Challenges
Technical
Optics
Weight
Battery Life
Bandwidth/Delay
Business
Price
Content
Application Platforms and Eco-system
26
30. Use Cases – Physical Objects
Design
Assemble
Warehouse/Logistics
Operate/Use
Maintain
Support. I See What You See
Survey
Education/Training
Execute a Procedure
30
31. Use Cases – Physical Spaces
Architecture/Interior Design
Navigation
Venue Experiences e.g. Museums
See Before You Buy
3-D Visualization in Space
Language Translation
…
31
32. Uses Cases – Immersive Experiences
Gaming
Social Virtual Experience
Movies
Theme Parks
Live Events – Every Viewer on Prime Seat
Virtual Tours
Before to travel
Real Estate
…
Holoportation
32
38. Example Use Cases
3D Virtual Design
Design prior to development
Visualize prior to purchase
Execute Procedure. Hands-free, 3D AR, Audio
assisted
Physical Object: Install, Operate, Maintain Example ;
Boeing Example ; Nokia Small Cell
Surgical/Medical Procedure
Pick and Pack
Surveys, Navigation
Always On
Self/Environment Monitoring, Compliance
Non intrusive Alerting, Audio output
Contextual Situation Awareness
Push Contextually Relevant Information
IOT Enabled Machine Visibility and Control
Multi party conference
I see what you see
Remote Expert, Customer Support, etc.
3D Non-real time Video for Training Example
Contextual Marketing
Larger Viewing Area, Non intrusive
Contextually relevant
Holo-portation. Real Time Communication
3D Virtual Travel
3D Immersive Entertainment
….
38
57. Questions to Ask
Can it enable meeting existing customer needs better ?
Can it enable meeting an unmet customer need?
Are there new customer segments that can now become attractive?
Can it improve monetization ? E.g. moving subscriber from free to premium offering?
Can it enable higher Productivity?
Can it enable higher Quality?
Can it enable greater Safety?
Can it help with Compliance requirements?
Directly from VR HMD/AR Smart Glasses and Technology building blocks
57
58. Resources
www.arglassesbuyersguide.com/
www.augmentedworldexpo.com/
Augmented Reality for Enterprise Alliance (AREA)
www.thearea.org/
Karl Guttag on Technology www.kguttag.com
Depth Sensing Technologies Comparison
http://www.ti.com/lsds/ti/sensing-products/optical-
sensors/3d-time-of-flight-sensors-technology-
comparison.page
58
Editor's Notes
We need to define something that is Valuable, Usable, Feasible and Viable.
Valuable: Deliver Value, benefit or utility to the customer.
Usable: The end-consumer is able to get that value easily and emotionally satisfying way.
Feasible: Feasible given all our constraints around resources, funding, internal capabilities, external capabilities and so on...
Feasible not just for our organization but also for all partners, who are involved in delivering that value.
All three combined should be viable. Viable for the customers, our organization, and partners e.g. channel partners.
This combination keep on changing. As new Technologies make feasible what was earlier not feasible, they Improve usability. As certain needs are met certain new one emerge making products more or less valuable.
The viability changes are our capabilities improves or prices drop.
As Product managers our challenge is to stay on top of them.
The objective on the talk is share developments, underlying technology components and some use cases in the space of AR/VR. What this is making feasible and what impact it has on usability.
For each us as Product Managers, to examine whether this can add or enhance our current offerings. Can it enables us to creates a competitive advantage in current market or enable us to enter new markets/customer segment by meeting unmet customer needs?
This is Product Mentor talk. Talk briefly about the Process of making that assessment. It that applies to any new technology.
So what is Augmented Reality/VR
A big component of this is a New Compute Interface. That redefines…
I call this Spatial User Interface
In your current actual physical view, that your eyes getting, when you are watching this on your PC/Mac or mobile. The PC/mobile is a 2D screen in your total physical view.
The 2nd key trait is Hands free.
And it looks something like this
This is how the interface looks like.
To understand it a little better….. Lets look at the Compute Interface Evolution
Lets look at the Compute Interface Evolution
Note that A single device can and usually does support multiple interfaces.
Augmented Reality is not just Spatial User Interface but it also includes applying voice interface, A 3D Spatial voice and GUI in space.
Lets understand the components behind this and how it is coming together. I will start with simple case and build it up. Lets start with Virtual Reality.
Lets start with the simpler virtual reality.
To get to 3D we need to two images one for each eye.
Stereoscopic 3D You need for two separate views one for each eye.
Lets see it in action.
You tube 360. glasses icon.
Show pan and tilt, movements.
Head Tracking –– Three sensors gyroscope, accelerometer and compass/magnetometer
Notice the forward or backward motion is not tracked in this case.
You need bring it close to your eyes to eliminate the actual view. And so you need lenses to zoom it.
Stop Sharing..
To take to the next level by
Show the lens/GearVR
The Display moves with your head… you get 360 degree view
To get a good experience you need high resolution and you need to refresh the frames. A minimum frame rate of around 60 frames per second is needed to users feeling sick. The current crop of VR headsets go way beyond this - Oculus is capable of 90fps, for instance, Sony's PlayStation VR manages 120fps.
Computation intensive for CPU and GPU => Lot of power and heat, battery life..
Start Sharing..
Describe in own words.. Switch to camera
One way to solve this is to connect to a more powerful processor. A PC or gaming console.
Add controller.
More immersive experience. The variation are smoother. The Resolution is higher, the Frame refresh rate is high (Sony Play Station can do 120 fps)
9 LED lights to track head movement
Challenge:
Tethered to Cable:
Cost. Sony Recent brought the price down to $400.
Oculus Touch is a set of wireless controllers, designed to make you feel like you're using your own hands in VR. There is also a matrix of sensors on each controller to detect gestures such as pointing and waving.
Eye tracking It's not available
An infrared sensor monitor's your eyes inside the headset so FOVE knows where your eyes are looking in virtual reality. The main advantage of this is to make depth of field more realistic.
And address the bandwidth challenge.
Position Tracking:
Oculus Development Platform :
Develop not just games, but also Apps, Social VR Experiences wherein individuals can create their Virtual Avatars and interact with others in Virtual Room. They can together talk, share a whiteboard, view a video or play 3D games all in the virtual room.
Content
A host of players are developing solutions and innovations in this space.
Large studio like Walt Disney to mom and Pop videography that record events like weddings are looking at creating content in 3D.
Now lets look at the more complex augmented reality.
It has been around from some time. We have seen it all.
Describe in own words. The idea is to augment the actual view with computer generated graphics.
For those of you outside of US, not familiar with the game … the orange and blue lines
This is simple case where there is one single POV where the camera is.
Pokemon Go
What is really happening is Anchoring/tracking to an object? A dollar bill in this case.
Take the feed of the camera and super-impose the image on an anchor.
This is called the Marker based tracking. Recognizing an image the app knows before hand.
Here notice that motion tracking is achieved due to the marker.
There is another approach Called Markerless Tracking but we not quite there which is
I did on my Samsumg Note.
Google is taking this forward with Tango.
Depth Sensing computer vision to enable mobile devices to detect their position relative to the world around them without using GPS or other external signals.
by illuminating an area with modulated IR light. By measuring the phase change of the reflected signal the distance can be accurately determined for every pixel in the sensor creating a 3D depth map
It store this map.
Lenovo Phab 2 Pro and the Asus Zenfone AR.
Priced : $500 not more expensive
An example of this..
Range is few meters upto 10-15 meters. And usually in indoor scenario.
Time of Flight ToF Depth Sensor 5Hz (time of flight camera).
3D time of flight operates by illuminating an area with modulated IR light. By measuring the phase change of the reflected signal the distance can be accurately determined for every pixel in the sensor creating a 3D depth map of the subject or scene.
It gives motion tracking and remembers it.
It is in 3D if you move but it still on 2D Screen in your view.
We need to a get to where the entire physical view is augmented.
Enter Smart glasses. Form factor
Now you have two images, one for each eye to get 3D.
Head tracking to see the computer generated graphics from different POVs.
They have full on board compute resources. Diplay sensors, battery, network connectivity. There is no cable.
They also include Sensors 6 DOF tracking. All other Sensors??
ODG R-9 enterprise $1800, R-8 $1000. Picture is R-8
ODG did not have gesture control but a track pad on the glasses, unlike the Hololens.
How is it coming together. It is coming together by Optics
Unlike in the phone where the camera stream is augmented here the actucal view is augmented.
MagicLeap.
basic tradeoffs between resolution, field of view, eye box, image quality, hardware weight/fit, aesthetic form factor,
FOV human eyes: 200x100 degree FOV
Flat: Google Glass gets 13 degree FOV, and Epson BT-300 gets 23 degree FOV with 1280x720 resolution.
Pros: Light, small, relatively affordable ($500–700ish).
waveguide hologram to progressively extract a collimated image guided by total internal reflection (TIR) in a waveguide pipe.
Pros: Potentially better FOV and resolution on medium sized devices.
Cons: Expensive (estimated $3k+), technology still actively in development, maybe black magic.
Cons: Very limited FOV and display, difficult to improve.
Curved:
Meta: “almost 90 degree FOV” and a 2560x1440 pixel display split between both eyes.
Pros: High FOV and resolution, relatively affordable ($900ish).
Cons: Bulky, low angular resolution, material quality risk
Even without this there are at
There are at least 40 different models of models across about 10 vendors?
These are mainstream at least not yet… for various reasons.
25+ AR Platforms with varying levels of functionality.
For example, in our example of the stool, we need to look up the details and order it,
Software Challenges:
Inconsistent buttons
What does pinch, swipe mean? Variations across
Scaling and managing smart glasses.
Most play in android.
Application authoring tools, Tools
Content : CAD designs, Object Recognition.
So After doing all this….
We are not quite there yet but that is where we moving towards.
Lets look at where it sits in the big picture…
Consumer vs. Enterprise play.
Here are some of he challenges.
For taking them mainstream and for wide spread adoption.
If you notice most of these challenges while critical for consumer adoption are not so crucial for Business adoption.
AR is getting traction on enterprises. I believe that AR will continue to get adopted by Enterprises. AR adopted will be driven by enterprises.
While VR is likely to be driven largely by Gaming and Events.
However there are a lot of smart people with lot of money are working to overcome these challenges. They are in fact competing to solve this before others.
E.g. ~ 4B venture funding has gone into the space over the last 3 years. + Investments from large players
100s of patents are filled every year over the last three years.
Google Tango/Day Dream,
Facebook: Oculus
Microsoft : Speculate has invested over 1B on Hololens with Wave guide Optics.
Apple: Rumored to launch AR in 2017.
Amazon: Filed application to FCC to test new devices.
On the Price do not let the price hold you back.
Can you get glasses that meet your specific needs at the required price.
What can you get on existing phones?
Can you offer it by providing the glasses yourself?
One can argue that why now we can wait till it matures. Sure.
The benefit of defining it now. Is to assess what incremental value it can add. How exactly you can advantage of the usability improvements. And even it turns out it is not feasible or viable. You have a view of what needs to change to make it feasible/viable. And then you can track and as soon as it meet. You can get into the market get the first move advantage.
And that is one possible way to get Competitive advantage.
Execute a procedure..
There are uses cases in the entire physical object life cycle.
Not just machines, but could also be organs e.g. doing a surgery.
Navigation
While driving – BMW use the windshield of the car as the combiner. It makes some of the challenges in smart glasses form factor like weight, battery life .. irrelevant.
It is not just normal driving …. Think of assistance it can provide during poor visibility conditions like fog.
Depth sensing to assist a Fireman to guide in smoke.
Example of aircraft assistance.
Shop 411.
The list is by no mean exhaustive. I still keep on learning and thinking about new use cases….
Hopefully, this triggered some creative juices for you to define your own uses for your own organizations.
Valuable Not just to customers but also to your organization, your partners e.g. channel partners.
Usable by end consumers
Feasible: Not just technical but also business. Feasible given your constraints around resources, funding, capabilities your and if you need partners then those of your partners.
They keep on changing as new Technologies make feasible what was earlier not feasible. Improve usability. As certain needs are met certain new one emerge or become more or less valuable.
To share developments in the space of Mixed Reality AR/VR. What is this making feasible and what impact it has on usability. For us as Product Managers. to determine if this add/enhance our offerings; enables us to creates a competitive advantage in product market mix or enable us to enter new markets/customer segment.
Enables us to meet unmet customer needs?
IMU A combination of Accelerometer, gyroscope and compass. Provide 6 degree of motion to track the head movement, position
The gyroscopes measures 360 degree rotation, accelerometers measure movement along the XYZ axes and magnetometers can determine orientation towards a magnetic field.
More Pictures ?
More Pictures ?
More Pictures ?
See if we can add more questions.
Marketing use cases?
If multiple use cases emerge then you have to prioritize.
Document the use case.
Look for the right Smart Glasses
Look for the right AR platform.
Is this something that is mature now or you need to wait and monitor?
Is there anything you can do to be prepared. Back end systems? Can ipad