1.
The rise of AR & VR:
why this time is different?
Vasily Ryzhonkov,
Director of Mobile Tech Center,
Skolkovo Foundation
vryzhonkov@sk.ru
http://mobile.sk.ru

2.
The Metaverse is coming…
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
Realtime networked 3D gaming was a
big step 19 years ago…
Source: Neal Stephenson,
Snow Crash, 1992
Source: Quake game by John
Carmack et al., 1996
…and now we got this.
Metaverse (coined in 1994 by Stephenson) is a collective virtual shared space, created by the
convergence of virtually enhanced physical reality and physically persistent virtual space, including the
sum of all virtual worlds, augmented reality, and the internet. (Wikipedia)

3.
Is this our future?
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
Fantasts have been dreaming about immersive
computing technologies for a long time…
Source: Minority Report, 2002 Source: Microsoft, 2015
…and in 2015 Microsoft presented it’s
vision of the future computing platform
(along with other concepts from
Facebook, Google, Sony, HTC, Valve …)

4.
VR/AR are the Next Big Thing
The Next Mega Tech Theme is Virtual Reality (VR).
• We believe virtual reality and augmented reality are the next mega tech themes through 2030.
• We liken the state of virtual and augmented reality today as similar to the state of mobile phones 15
years ago.
• It likely will take a decade before mainstream adoption as necessary improvements in displays and
applications as well as lower pricing are needed to drive demand.
• The first phase will be virtual reality, which we believe will start over the next several years, and is the
primary focus of this report; the second phase will be augmented reality.
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

5.
Technology Definitions
Augmented Reality
Augmented reality, or AR, is when real reality, or RR, is enhanced with virtual
images that appear to coexist with the real world.
Virtual Reality
Virtual reality, or VR, is when you’re immersed in a purely virtual world
5
Mixed Reality (MR) or Diminished Reality, refers to a system that combines
real and virtual objects and information using augmented virtuality, which is
combines some reality with virtual world.
Mixed Reality
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

6.
Technology Definitions
What is Total Immersion?
According to Wikipedia, Immersion into virtual reality is a perception of being physically present in a non-
physical world. This combines the sensory perceptions of the human mind to give a surreal impression of
living in another world, constructed of Virtual or Augmented reality. Additionally, Virtual reality glasses
can produce a visceral feeling of being in a simulated world, a form of spatial immersion called Presence.
According to Oculus VR, the technology requirements to achieve this visceral reaction are low-latency and
precise tracking of movements
6Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

7.
Ontology of AR / VR
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
Source: “Mediated Reality with implementations
for everyday life”, Steve Mann, 2002
Source: "Augmented Reality: A class of displays
on the reality-virtuality continuum“, Milgram,
Paul; H. Takemura; A. Utsumi; F. Kishino, 1994
Father of Augmented Reality,
Steve Mann

8.
0 WAVE: Infancy
Mirror stereoscope: Sir
Charles Wheatstone
1840
Stereoscope: An optical illusion instrument, giving
depth of field through two pictures of the same object
Sir David Brewster created
a new stereoscope by
utilizing prismatic lenses
1841 - 1860
Viewing 3D movies through stereoscope at home
1922
First 3D film
produced in
Hollywood: The
Power of Love
1923 - Present
Mainstream 3D Cinema begins
8
Stereoscope: Sir
David Brewster
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

9.
1st WAVE: Inventors & Dreamers
1957
Sensorama: First VR machine
developed by Morton Heilig,
gives visual and sensory
perceptions of virtual reality,
including vibrations, smell
and touch.
1930 1965
Link trainer: Invented by
Edwin Link for training
military pilots, using
simulation
“Ultimate Display”, a VR
helmet with two CRT
screens, was created by
Ivan Sutherland. The
two CRT screens
enabled the viewer to
experience the 3D
images
9Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

10.
2nd WAVE: Industrial & Military Applications
1966-68
Thomas A. Furness III developed
the first VR head mounted
display cockpit for the US Air
Force, with Bob Sproull.
1980- Present NASA: Virtual Visual Environment
Display, head mounted unit for VR
graphics and simulation using LCD
display and wide angle lens.
10Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
General Electric Corporation built one of the first computerized
flight simulators. It provided 180-degree FOV (Field-of-View) by
using three screens surrounding the training cockpit.
1972

11.
3rd WAVE: Industrial Applications & Gaming
DHL: Using Augmented
and Virtual Reality, DHL can
optimise parcel and load
handling, customer delivery
efficiency and provide
detailed tracking
ESI: Improved data handling,
warehouse management and
capacity forecasting using AR
and VR technologies
Using an iPad, with an
application specially
developed by Alstom
Transport, all maintenance
tasks are described with
precision superimposing
in real time a 3D virtual
model on the operator’s
perception of reality
11Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
Nintendo Virtual Boy :
1994, price - $180.
Sega VR: 1994,
prototype

12.
4rd WAVE: Next computing platform
The 4th wave of virtual and augmented reality began, when
Facebook bought, the VR pioneering company Oculus for $ 2
Billion US Dollars.
The acquisition sparked a market speculation and generated a
renewed interest in VR and AR technology. The Google Trend
graph depicted in adjacent, shows the interest in the
technology over time since 2014.
Google Glass, launched in 2013 is another example built on
(optical head-mounted display ) the next generation
computing platform using Ubiquitous computing for
displaying real time AR information through lens.
12
Google Trends: Virtual Reality
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

13.
Predictions and hype present and in the past
Source: Gartner (2008)
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
Source: Gartner (2015)
The only outcome I can make is …predictions are always wrong…but they can
indicate media hype and technological advances

14.
Predictions and hype present and in the past
Source: Frost & Sullivan, 4th Wave in:Bauer (1996), OVUM in Leston et al. (1996),
Machover (1996)
“Are these expectations
realistic or do they result from
the initial euphoria of an
innovative technology?”, -
Economist, 1996
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

15.
7 Failed Virtual Reality Technologies
The Sensorama : Morton
Heilig, inv - 1957, patent -
1962
The Sword of Damocles :
Ivan Sutherland , 1968
Nintendo Virtual Boy :
1994, price - $180.
Power Glove : Nintendo,
1989, price - $75
The Virtusphere : Ray
and Nurulla Latypov,
1995, price - $100K
Omnidirectional Treadmills
VRML, 1995
VRML was billed as a 3-
D alternative to HTML.
The idea was that users
could interact freely with
3-D worlds on the
internet, described by
text and interpreted by
modeling software.
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

16.
Why AR & VR didn't get broad adoption?
• Technical imperfection
• Cost
• Bulky & Bad design
• Failed to deliver value to the end user
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

17.
What's now and why 2014th is the cornerstone?
Facebook’s Oculus Rift and Sony Playstation.VR is only beginning…
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

18.
What's now and why 2014th is the cornerstone?
Critical mass of elements has come together
Organizational and economic:
• critical mass of companies is working together
• price and affordability
• ergonomics and design (not bulky)
Technical:
• flat-panel displays
• cell phone panels (good megapixels)
• optics (wide fov)
• mobile CPUs & GPUs
• batteries and power management
• wireless
• gyroscopes and accelerometers
• cameras
• projectors
Technical:
• computer vision
• tracking hardware
• content 3d games
• latency***
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

19.
AR & VR Ecosystem
19Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

20.
AR & VR Ecosystem : Cont.
20Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

21.
AR & VR Ecosystem : Cont.
DISPLAY HARDWARE: [HMD]: AlterGaze, AntVR,
FOVE, AVEGANT, HTC, Oculus, Sony, Panasonic,
VRVANA, Razer, SENSICS, Immersion VRELLA, Light
& Shadows & Mindmaze.
[Wireless/ Mobile VR]: Moar, LG, ZEISS, Wearality,
USENS, Homido, Samsung, GearVR & VR One.
[GAMING]: Activision, Blizzard, CCP, Epic Games, nDreams, Otherworld,
Reload, Owlchemy Labs, Vertigo Games & VR Bits.
[CAVE]: Visbox, Worldviz, Christie and mechdyne.
21Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

22.
VR Ecosystem
22Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
Source: Greenlight VR, UploadVR
Number of VR companies Investments in VR
Top 10 companies in VR based on Total Funding Raised

23.
VR Ecosystem
23Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
Source: Greenlight VR, UploadVR
Total funding raised by all VR companies
VR Investment landscape

24.
Russian Landscape
There are more than 60 companies in Russian AR/VR market.
Mostly these companies are digital studios. However, there are high-tech companies with proprietary technology
developed in-house.
24Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

25.
Russian Landscape: Cont.
Augmented reality companies:
25
За 2014 год произошел резкий всплеск
образования компаний,
занимающихся интерактивными
технологиями (40% компаний).
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

26.
Russian Landscape: Cont.
Virtual reality companies:
26Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

27.
Russian Landscape: Cont.
AR / VR continuum: controllers / 3D visualization & 3D content creation / haptic & sensory feedback / images
& video search and analysis / gesture recognition, computer vision etc.
27Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

28.
Current Market Analytics :- Revenue Users and Investments
Given the scale of customer adoption, it has
given rise to steady revenue over time, totalling
over 62 $ Million US dollars by 2017 with AR
revenue share consisting of 600 $ Million US
Dollars by 2017. The agencies suggest that a total
of revenue of 160 $ Million US Dollars by 2018,
with head mounted display sales leading to 117 4
Million US Dollars.
28Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

29.
Current Market Analytics :- Market Share
Different analytical companies show different
estimates. Piper Jaffray suggests that VR
market can grow up to $7 bln by 2025; some
agencies assess market as $150 bln by 2020
consisting of AR and VR market share.
Upon further analysis, we see that market will
consist of VR Hardware, VR games, VR Film &
Concerts, VR amusement parks, VR
Entertainment, VR speciality products (such as
adult content), VR industrial applications.
29Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

30.
Market Analytics : Revenue generation and adoption
Piper Jaffray report:
“We believe technology advances over the next 3-5 years
will continue to drive down the cost of these technologies
and enable significant improvement in tracking and
rendering. We expect CPUs and GPUs of computers to
continue along Moore’s Law which should enable
cheaper computers to eventually run wired headsets
headsets. We expect VR will gradually drop from an all
in entry cost of $1000+ (VR headset $400 + high
powered computer $1,000) to a more reasonable $500+
(VR headset $250 + computer $300) for VR headsets
that offer high levels of immersion”
Conclusion: HMD price will be key driver of the mass
adoption.
30Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

31.
Market Analytics : Revenue generation from HMD
Based on the information above, we can conclude that VR HMD will drive the industry forward.
However, content will be king. That’s why a lot of VR companies invest heavily in VR studios, create SDK
for developers. And recently Oculus Rift announced that hundreds of games are in the process of the
development and thousands of developers are involved in this process.
31Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

32.
Gaming market dissection
During the third quarter of 2014 , KZero, a
market analytics firm conducted a market
research within the gaming section of VR
sectors.
It was found that RPG style games were most
popular amongst gamers who are older than 30
years old, with simulation and exploration type
games being popular between 20-30 year old
gamer
Based on the red colour depiction within
Simulation, it can be inferred that majority of the
games are in late developmental or in released
stage, whilst RPG (Role Playing Games) are in
early development or alpha stages.
32Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

33.
Analysis Conclusion
1. Strong correlation can be seen with other
market technologies like gaming, 3D, motion
tracking, which are interconnected.
2. It has been estimated that, there will be 170
million active users in 2018, accounting to the
total gaming population of 5.2 billion at large.
3. Gaming industry is dominated by software
sector and is believed to remain that way
4. Most opportune and profitable moment in
cycle is between 2017 and 2018, for technology
will be developed and available to bigger mass,
leading to a predominant early majority
generating the highest fixed return per user.
Highest return on adoption is probably
somewhere near the start of next year,
5. To conclude, Augmented Reality is expected to
rise faster than VR, due less complexity,
increased adaptability, more interoperability,
less motion sickness and simulation sickness
33Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

34.
VR/ AR Market drivers and projections
34
1. An estimated 3000+ million
mobile subscribers are expected in
2020, having a penetration rate of
51% at a global average.
2. This provides Augmented Reality,
a development scope in games,
enterprise application and location
based services.
3. It has been estimated that USA,
Japan and SE Asia will be the most
profitable markets for AR.
4. The key market drivers for AR are,
mobile devices, app stores and
gaming apps.
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

35.
Challenges and barriers of AR & VR
Economic and organizational risks of Virtual reality:
• Physical Risk. Any discrepancy between what the user sees and feels
could lead to motion sickness.
• Security Risks. For example, if users are able to modify the appearance
of their avatars, they could potentially impersonate other users.
• Privacy Risks. A virtual environment can dramatically expand the scope
of potential privacy violations because every single behavior in a virtual
environment can be tracked, and every element of a virtual
environment can be manipulated.
• Price. Many experts think that the limiting factor for virtual reality is
going to be the price. The reality is a bit more complicated.
• Value for customers. Consumer appetite for VR is hard to predict and
need to be stimulated.
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

36.
Challenges and barriers of VR
There are four really hard problems that if solved
would make VR far more convincing:
• Tracking
• Latency
• Stimulating the human perceptual system to produce
results indistinguishable from the real world
• Virtual reality sickness
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
Barriers from technological perspective:
• GPU and CPU horsepower
• Input
• Tracking
• Computer vision
• Display
• Audio
• Haptics
• Sense of empathy
• Etc…

37.
Use case: GPU horsepower needed for mobile VR
Goal is to calculate GPU horsepower needed for low-end but realistic mobile VR immersive HMD
Information given:
1. Display Resolution: 1080p
2. Frames per second: 60-90
3. Depth of resolution: 24 bit
4. Texture information: 12
million per scene
5. Latency: <20 ms
6. Pixel Density: 300 dpi
Calculation:
1. The GPU needs 50 bytes per polygon, which then is
multiplied for 12 million polygons, would be 600
Megabytes per second, with 60 frames per second.
2. In order to calculate the Bandwidth, it will be : 600
megabytes x 60 frames/sec = 36 GB/sec [Pass 1]. For 90
frames/sec it would be 54 GB/sec
3. According to OpenGL and DirectX best practices, a
minimum of 3 passes on each polygon must be made.
This is done in order to draw the polygon, emulate its
animation properties and to display its texture
37Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
Tesla GPU
NVIDIA
Pascal GPU

38.
Use case: GPU horsepower needed - Cont.
Information given:
1. Display Resolution: 1080p
2. Frames per second: 60-90
3. Depth of resolution: 24 bit
4. Texture information: 12 million per
scene
5. Latency: <20 ms
6. Pixel Density: 300 dpi
Calculation:
4. Therefore, the bandwidth required by the GPU would be 36 GB/sec x
3 = 108 GB/sec. For 90 frames it would be 54 GB/sec x 3 = 162 GB/sec
5. Additional considerations: Texture characteristics and attributes, such
as shining, shading, colour and material need to be analysed and
incorporated within these parameters
6. In order to rasterize, 1 polygon, it would need around 100 floating-
point operations. Therefore, for 12 million polygons, it would be:
12,000,000 x 100 flops x 60 frames/sec x 3 raster passes,
it would equate to around: 216 GFLOP/ sec
For 90 fps – 324 GFLOP/s
Based on the information above, NVIDIA have suggested GTX 970 to be
the capable GPU according to the requirements.
GTX 970 has 196+28 GB/sec – 3494 GFLOP/s – 145 Wt - - $329 – June,
2014
Top is GTX Titan X – 336 GB/sec – 6144 GFLOP/s – 250 Wt - $999 – 1kg –
Mar, 2015 38Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
CPU vs GPU Performance

39.
Use case: Power consideration.
Information given:
1. Power Required: 145 Watts per
Hour
2. Duration: 1, 2, or 8 hours per cycle
3. Battery Type: Lithium Ion
39Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
Therefore, energy stored in the battery for scenario 1 would be: 145
Wt*h
Scenario 2: 290 Wt*h
Scenario 3: 1160 Wt*h
For commercial implementation, Lithium Ion gives best performance
and is supported by wide industry standards.
The typical energy density stored in a mobile phone battery is
between 0,9 – 2,3 MJ/L. This equates to 740 Wt*h/L.

40.
Use case: Battery required to power GPU for realistic scene
40Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
Energy density of Lithium Ion batteries over the years, Battery University, 2014
Scenario 2:
Volume = 0,000391 m3
Equal dimensions = 0,073m
Scenario 3:
Volume = 0,001567 m3
Equal dimensions = 0,12 m
Outcome:
1. For mobile VR system with realistic graphics
working for 1 hour we will need battery with 6
cm x 6 cm x 6 cm dimensions.
2. For mobile VR HMD working at least 8 hours
we will need a battery with 12 cm x 12 cm x 12
cm.

41.
Complexities within VR & AR
IMAGE DISPLAY
IMAGE RENDERING
DRIVER PROCESSING on CPU
SPATIAL AUDIO
PIXEL PERSISTENCE
HEAD TRACKING
HAPTICS
INPUT
BODY TRACKING
Graphics
Sub-
system
VR
Headset
Other
Peripherals
Based on the data obtained from AMD, with Oculus being the benchmark
standard, it is evident from the information table above, that a ordinary
gaming PC would be unable to suffice the demanding computations for
this technology.
Additionally, in order to improve the visual experience the whole sub-
system, starting from input control to image display needs to be optimised
for minimal latency and maximum throughput.
This includes, rendering in 2D, 3D and immersing displays with texture
based rendering.
41Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

42.
CPU and GPU Performance over time
The adjacent CPU/GPU graph suggests, using Moore’s Law,
coupled with Nanotechnology together with advancements
in parallel computation has boosted the overall performance
of GPU in relation to graphic intensive computation.
Based on both graphs, it can be inferred that price of
computing will do down significantly, whilst the performance
gains are expected to be exponential over time.
According to the computation by AI Impact, it suggests that the
price of a gigaFLOPS falls by an order of magnitude roughly
every four years (14 orders of magnitude in 54 years is 3.9 years
per order of magnitude).
Since 1997, each order of magnitude only took three years (5.7
orders of magnitude in 18 years).
42Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

43.
Key outcome of the GPU and Battery case
GPU horsepower case
1. There are cards that already can satisfy minimum
requirements for realistic immersive VR systems. But
they will work only with high-end laptops and PC for
the next 5 years.
2. According to current trends the GPU horsepower will
be increasing, but the power and weight will also be
increasing – 250 Wt - $1000 price and 1 kg weight are
limiting factors for mobile VR.
3. This leads us to the total cost of VR system for the
end user (gaming market). It will consist of laptop +
high-end GPU + HMD + content = $2000 + $1000 +
$ 1500 + $1000 = $5500 per year per 10 pieces of
content. A bit expensive, isn’t it?
4. Mobile VR will be developing with low-end graphics
and will not give opportunity to be totally immersed.
5. Mixed systems will be developing where
computational power will be in the backpack of the
gamer
Battery case
1. Batteries will still be the problematic part of
the equation. The energy density/weight
and energy density/volume ratios are
progressing not so fast as energy
density/cost ratio. It’s very critical issue for
mobile VR systems and will not be solved
for the next 5-10 years according to current
trends in technology.
43Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
The only way I see is rethinking of
computing architectures. What AMD,
Nvidia, and Intel already are doing….

44.
Progress within VR & AR
“LiquidVR is a silky smooth player” –
The Guardian
“GameWorks VR: Smarter graphics for
virtual reality” - CNET
Recently, AMD has launched Project: LiquidVR®
dedicated to VR and AR technology. According to
AMD, GPU software and hardware sub-systems to
tackle the common issues and pitfalls of achieving
presence, such as reducing motion-to-photon latency
to less than 10 milliseconds.
In other developing events, Intel has partnered
with Oculus Rift to form VR alliance in
association with Altspace VR to establish a
common platform for better user experience.
Additionally, 6th generation Intel processors
have incorporated Intel RealSense®
Technology which is a gesture based Human-
Computer Interaction model to process and
compute AR/VR information.
Microsoft has also made extensive efforts to
incorporate VR and AR techno in the latest
release of Windows 10, and are working on
releasing the HoloLens (VR HMD) device logy in
2015.
44Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

45.
Challenges and barriers of AR
These are AR technological barriers:
• Wide-field-of-view see-through display
• Input
• Mobility
• GPU and CPU horsepower
• Battery
• Room-scale tracking
• Walk-around tracking
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
The ODG Smart Glasses
Source: ODG Group

46.
Critical Success Factors : Vision 2020
Cheaper
cost of
devices
Wider
availability
of content
Elimination
of motion
sickness
Portability
Increased
battery life
Affordable
Manufacturing
Affordable
Price
Mobility
Vision
Immersion
Usability
Flexibility
Wearability
“AR needs to become more
mobile than mobile to
compete,“
– Binocular IO
“While internal market
competition is inevitable, the
fight that really matters is
between AR/VR’s installed base
and the huge incumbent
smartphone and tablet market”
– DIGI Capital
46Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

47.
What are key areas for innovation in AR & VR?
VR will kick off graphics into hyperdrive :
• 90 hz stereo, Can't miss frames
• 110 fov
• Resolution of 10k x 10k
• High rendering quality
• New rendering techniques
• GPU performance 2 to 3 orders of magnitude
• Transmission links
• Display ( 100 megapixels and more, thousands of pixels per inch, HDR)
• Rethinking art and animation technology
• Optics
• Rendering engines and content creation platforms
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

48.
What are key areas for innovation in AR & VR?
• Wide field of view (immersion, peripheral cues, 100-110 degrees)
• Adequate resolution (problematic for VR due to wide FOV, one-seventh the pixel density per
degree of a wide-screen TV - 26p, 1080p works, more is better)
• Low pixel persistence (necessary to avoid blurring with eye motion, no more than 3 ms)
• A high enough refresh rate (to avoid flicker with low persistance, 95 Hz seems to be sufficient)
• Global display (illuminates all pixels simultaneously, avoid motion-induced compression,
stretching, and skewing)
• Optics (only one or at most two lenses per eye, not enough to allow simultaneous control of
focal length, viewing distance, size, distortion, various forms of aberration)
• Mobility
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

49.
What are key areas for innovation in AR & VR?
• Optical calibration ( the human visual system is amazingly sensitive to deviations, many
problems are impossible to identify until calibration is done )
• Rock-solid tracking (must support translation (position in x, y, z) as well as orientation, position
with at least milimeter accuracy, orientation with at least quarter-degree accuracy, volume at
least 1,5 meters on a side)
• Low latency (20 ms motion-to-last-photon works, 25 ms may be good enough)
• Resolution (The math for resolution together with wide fields of view is brutal; divide 1K by 1K
resolution – about the best an affordable head mounted display is likely to be able to do in
the next year – into a 100 - degree FOV and you get a display with less than 1/50th the pixel
density of a phone at normal viewing distance)
• Input
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

50.
Implications and how it will change any aspect of our life
VR will influence all aspects that it touches:
• Presentation
• Art and creation methods
• Ergonomics
• Productivity (the way we work)
• Locomotion
• Social interaction
• Total immersion and mixed reality
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

51.
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
What is total immersion? Mixed reality as a new media…

52.
Total Immersion Required Characteristics
Unique characteristics of immersive virtual reality:
• Head-referenced viewing provides a natural interface for the navigation in three-dimensional
space and allows for look-around, walk-around, and fly-through capabilities in virtual
environments.
• Stereoscopic viewing enhances the perception of depth and the sense of space.
• The virtual world is presented in full scale and relates properly to the human size.
• Realistic interactions with virtual objects via data glove and similar devices allow for manipulation,
operation, and control of virtual worlds.
• The convincing illusion of being fully immersed in an artificial world can be enhanced by auditory,
haptic, and other nonvisual technologies.
• Networked applications allow for shared virtual environments.
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

53.
What is total immersion? Mixed reality as a new media…
Technical requirements:
• Face tracking
• Eye tracking
• Head tracking
• Hands tracking
• Body tracking
• Position tracking
Apply that to virtual avatar:
• Input
• Audio
• Interaction
• Haptics
• Perceptual research :
psychology and other
human factors
Multimodal interactions are key to VR:
• Feedback loop (see the object, touch it, sense it)
• Correspondence between insights
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

54.
AR & VR as Next Computing Platform
• Drives the perceptual system as its built to be driven
• Different and more powerful mode of interaction
• Proper superset of all existing means of communication
• In the limit, its as big as reality ... or bigger
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

55.
What the future of VR & AR will look like?
● Virtual reality will be most popular in the gaming market, because of the high level of immersion
possible in VR, as well as the high demand for interactive experiences.
● Augmented reality will be most successful in the enterprise market, for applications in logistics,
engineering, and automotive.
● Applications such as education, travel, and design are served well by both AR and VR, and success
depends more on specific needs than general application.
● New content mediums, such as interactive VR video, are becoming more popular and catching the
attention of content creators as a potentially viable medium. Existing software solutions, such as CAD
software, can be outfitted to support AR and VR devices, and open a new approach to using the
software.
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

56.
What's next? Possible roadmaps
Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru
Scenario 1.
• VR will be the first in mass adoption.
• The roadmap for VR is likely to be pretty straightforward: gaming market,
entertainment market.
Scenario 2.
• AR will be second and after 2020.
• Industrial applications of AR find it’s slow way to the market overcoming
technical & organizational barriers.
• Information glasses succeed.
Scenario 3.
• Console AR revolutionizes living-room gaming.
• Next step – whole new computing platform. Laptops and PCs are replaced
by HMD.

57.
The future of VR & AR
“The return of virtual reality: 'this is as big an opportunity as the internet” –
The Guardian
“The Future Of Virtual Reality Might Be Present” –
The Forbes
“Another use case for VR photography could be news journalism”
– ABC News
“Virtual Reality is inevitable” - Palmer Luckey
“Ten years ago, this capability was purely in the realm of futurists and
seemed like a dream. Now, it feels like a very real possibility that will
enable us to experience nature, architecture, history and culture in new
and meaningful ways, and possibly lead to a new renaissance of art and
discovery”. – The Washington Post
“The Future Is Right In Front Of Our Eyes” -
TechCrunch
“A natural next step for VR is full-immersion home
entertainment” - TechCrunch
As we progress into the 21st Century. Augmented
and Virtual Reality will play an integral part of our
lives, helping and assisting us in every way
imaginable.
Given the significant advancements within the
same, this technology is set to become multi-
faceted stream of communication sectors, from
news media to transport and logistics to name a
few.
57Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru

58.
Our Team 58
Vasily Ryzhonkov,
Director of Mobile Tech Center,
Skolkovo Foundation
vryzhonkov@sk.ru
http://mobile.sk.ru
Joshua Levy
Intern at Mobile Tech
Center, Skolkovo
Foundation
58Vasily Ryzhonkov : vryzhonkov@sk.ru : +7 (916) 004 03 15 : http://mobile.sk.ru