Se ha denunciado esta presentación.
Utilizamos tu perfil de LinkedIn y tus datos de actividad para personalizar los anuncios y mostrarte publicidad más relevante. Puedes cambiar tus preferencias de publicidad en cualquier momento.
Computational Near-eye Displays with Focus
Cues
Gordon Wetzstein
Stanford University
SID Display Week 2017
www.computation...
Personal Computer
e.g. Commodore PET 1983
Laptop
e.g. Apple MacBook
Smartphone
e.g. Google Pixel
AR/VR
e.g. Microsoft Holo...
A Brief History of Virtual Reality
1838 1968 2012-2017
Stereoscopes
Wheatstone, Brewster, …
VR & AR
Ivan Sutherland
VR exp...
Ivan Sutherland’s HMD
• optical see-through AR, including:
• displays (2x 1” CRTs)
• rendering
• head tracking
• interacti...
Where we are now
IFIXIT teardown
Tutorial Overview
• conventional, fixed-focus near-eye displays
• focus cues & the vergence-accommodation conflict
• advan...
Magnified Display
1
d
+
1
d'
=
1
f
d
d’
f
• big challenge:
virtual image
appears at fixed
focal plane!
• no focus cues
Duane, 1912
Nearestfocusdistance
Age (years)
8 16 24 32 40 48 56 64 72
4D (25cm)
8D (12.5cm)
12D (8cm)
Importance of Focus...
4D / 25cm Optical Infinity
Normal vision
Nearsighted/myopic
Farsighted/Hyperopic
Presbyopic
Focal range (range of clear vi...
Relative Importance of Depth Cues
Cutting&Vishton,1995
The Vergence-Accommodation Conflict (VAC)
Real World:
Vergence &
Accommodation
Match!
Current VR Displays:
Vergence &
Accommodation
Mismatch
Accommodation and Retinal Blur
Blur Gradient Driven Accommodation
Blur Gradient Driven Accommodation
Blur Gradient Driven Accommodation
Blur Gradient Driven Accommodation
Blur Gradient Driven Accommodation
Blur Gradient Driven Accommodation
Top View
Real World:
Vergence & Accommodation Match!
Top View
Stereo Displays Today (including HMDs):
Vergence-Accommodation Mismatch!
Screen
VR Displays with Focus Cues
1. Adaptive and Gaze-contingent Focus
Magnified Display
Display
Lens
Fixed Focus
1
d
+
1
d'
=
1
f
d
d’
f
Adaptive Focus
Magnified Display
Display
Lens
1
d
+
1
d'
=
1
f
actuator  vary d’
Adaptive Focus
Magnified Display
Display
Lens
focus-tunable
lens  vary f
1
d
+
1
d'
=
1
f
Adaptive Focus - History
• M. Heilig “Sensorama”, 1962 (US Patent #3,050,870)
• P. Mills, H. Fuchs, S. Pizer “High-Speed I...
Padmanaban et al., PNAS 2017
Padmanaban et al., PNAS 2017
Padmanaban et al., PNAS 2017
Padmanaban et al., PNAS 2017
Padmanaban et al., PNAS 2017
Padmanaban et al., PNAS 2017
Padmanaban et al., PNAS 2017
at ACM SIGGRAPH 2016
EyeNetra.com
at ACM SIGGRAPH 2016
participants of the study, 152 total
EyeNetra.com
Participants - Prescription
Padmanaban et al., PNAS 2017
n = 70, ages 21-64
vergence
accommodation
Conventional Stereo / VR Display
vergence
accommodation
Removing VAC with Adaptive Focus
Follow the target with your eyes
4D
(0.25m)
0.5D
(2m)
Task
Stimulus
Padmanaban et al., PNAS 2017
Accommodative Response
RelativeDistance[D]
Time [s]
Stimulus
Accommodation
n = 59, mean gain = 0.29
Padmanaban et al., PNAS 2017
Accommodative Response
RelativeDistance[D]
Ti...
Stimulus
Padmanaban et al., PNAS 2017
Accommodative Response
RelativeDistance[D]
Time [s]
Stimulus
Accommodation
Padmanaban et al., PNAS 2017
Accommodative Response
RelativeDistance[D]
Time [s]
n = 24, mean gain ...
Padmanaban et al., PNAS 2017
Do Presbyopes Benefit from Dynamic Focus?
Gain
Age
Padmanaban et al., PNAS 2017
Do Presbyopes Benefit from Dynamic Focus?
Gain
Age
conventional
Padmanaban et al., PNAS 2017
Do Presbyopes Benefit from Dynamic Focus?
Gain
Age
conventional
dynamic
Gaze-contingent Focus
• non-presbyopes: adaptive focus is like real world, but needs eye tracking!
HMD
lens
micro
display
...
Gaze-contingent Focus
Padmanaban et al., PNAS 2017
Gaze-contingent Focus
Padmanaban et al., PNAS 2017
Gaze-contingent Focus
Padmanaban et al., PNAS 2017
at ACM SIGGRAPH 2016
Gaze-contingent Focus – User Preference
Padmanaban et al., PNAS 2017
VR Displays with Focus Cues
2. Monovision
Monovision VR
Konrad et al., SIGCHI 2016; Johnson et al., Optics Express 2016; Padmanaban et al., PNAS 2017
Monovision VR
Konrad et al., SIGCHI 2016; Johnson et al., Optics Express 2016; Padmanaban et al., PNAS 2017
• monovision d...
VR Displays with Focus Cues
3. Multiplane Displays
Multiplane VR Displays
• Rolland J, Krueger M, Goon A (2000) Multifocal planes head-mounted displays. Applied Optics 39
• ...
Multiplane VR Displays
• Rolland J, Krueger M, Goon A (2000) Multifocal planes head-mounted displays. Applied Optics 39
• ...
VR Displays with Focus Cues
4. Light Field Displays
Light Field CamerasLight Field Stereoscope
Huang et al., SIGGRAPH 2015
Backlight
Thin Spacer & 2nd panel (6mm)
Magnifying Lenses
LCD Panel
Light Field Stereoscope
Huang et al., SIGGRAPH 2015
Near-eye Light Field Displays
Idea: project multiple different perspectives into different parts of the pupil!
Target Light Field
Input: 4D light field for each eye
Multiplicative Two-layer Modulation Input: 4D light field for each eye
Multiplicative Two-layer Modulation Input: 4D light field for each eye
Multiplicative Two-layer Modulation Input: 4D light field for each eye
Multiplicative Two-layer Modulation
Reconstruction:
for layer t1
Tensor Displays,
Wetzstein et al. 2012
Input: 4D light fi...
Traditional HMDs
- No Focus Cues
The Light Field HMD
Stereoscope
Light Field Stereoscope
Huang et al., SIGGRAPH 2015
Traditional HMDs
- No Focus Cues
The Light Field HMD
Stereoscope
Light Field Stereoscope
Huang et al., SIGGRAPH 2015
Traditional HMDs
- No Focus Cues
The Light Field HMD
Stereoscope
Light Field Stereoscope
Huang et al., SIGGRAPH 2015
Traditional HMDs
- No Focus Cues
The Light Field HMD
Stereoscope
Light Field Stereoscope
Huang et al., SIGGRAPH 2015
Tensor Displays
Wetzstein et al., SIGGRAPH 2012
Vision-correcting Display
iPod Touch prototypeprinted transparency
Huang et al., SIGGRAPH 2014
prototype
300 dpi or higher
Huang et al., SIGGRAPH 2014
VR Displays with Focus Cues
5. Maxwellian-type Displays
Maxwellian-type Near-eye Displays
• eyebox of display is a pinhole  very large depth of field (no
retinal blur cue)
• exi...
Maxwellian-type Near-eye Displays
Accommodation-invariant Near-eye Displays
Konradetal.,SIGGRAPH2017
Accommodation-invariant Near-eye Displays
Konrad et al., SIGGRAPH 2017
Accommodation-invariant Near-eye Displays
Konradetal.,SIGGRAPH2017
Accommodation-invariant Near-eye Displays
Accommodation-invariant Near-eye Displays
Konrad et al., SIGGRAPH 2017
Other Upcoming Work at SIGGRAPH 2017
“Focal Surface Displays”, Matsuda et al.
“Accommodation and Comfort in Head Mounted D...
Summary
• focus cues in VR/AR are challenging
• adaptive focus can correct for refractive errors (myopia, hyperopia)
• gaz...
Overview of Optical See-through AR Displays
slidebyBernardKress
Consumer Applications – Prescription Glasses
Google Glass
small!
Google Glass
Pepper’s Ghost 1862
Microsoft Hololens – Waveguide with Holographic Optical Elements
Oculus DK2 (115 deg)
FOV (diag) and FOV location in some of the current products
Sony Morpheus (90 deg)
Sony HMZ-T3 (51deg...
FOV and angular resolution for current productsAngularresolution(pixels/deg)
FOV (diag, deg)
Eye resolution limit (1.2 arc...
There are two main optical HMD architectures
slidebyBernardKress
Opaque Bird bath Free space
Waveguide
EPE
Lightguide
There are 5 main head-worn optical platform architectures
developed i...
Oculus DK2
Sony HMZ
Vuzix
Occlusion
Huge FOV
Large eyebox
Opaque Bird bath Free space
Waveguide
EPE
Lightguide
slidebyBern...
slidebyBernardKress
Oculus DK2
Sony HMZ
Vuzix
Vuzix M100
MyVu
…
Occlusion
Huge FOV
Large eyebox
Partially occluded
Small FOV
Medium eye box
Op...
Most of the early occlusion HMDs are based on the combination of a
transmission lens and a 45 degrees mirror (either TIR o...
“Bird bath” optical architectures
Google Glass
RockChip Ltd
ITRI
Good see through
Small FOV
Medium eye box
Opaque Bird bat...
Google Glass
slidebyBernardKress
Laster Sarl
ODA Labs
…
Bug eye optics
Large FOV
Medium eyebox
Temple projector
Opaque Bird bath Free space
Waveguide
EPE
L...
The notion of « Bug Eye » in HMD combiner opticsslidebyBernardKress
This is what we mean with the « bug-eye » concept
slidebyBernardKress
ORA Associates (US)
Reflective “bug eye” combiner
Binocular version
Laster Sarl (Fr)
Reflective combiner (pupil forming
ar...
Canon Ltd
Motorola HC1
Kopin Golden
eye
Distorted see-through
(or opaque)
Medium FOV
Medium eyebox
Without complement piec...
Canon Ltd
Motorola HC1
Kopin Golden
eye
E-Magin Optics
Fraunhofer
Distorted see-through
(or opaque)
Medium FOV
Medium eyeb...
TIR / free-form surfaces combiner (including eye tracking)
Example with
complement piece
for see through
slidebyBernardKre...
Epson Ltd
Moverio 1 & 2
…
OK see through
Medium FOV
Medium eyebox
Curve coated reflector combiner
Opaque Bird bath Free sp...
Epson Corp, Moverio (first and second generations)
3 collimation lenses, prism injector, 6 TIR bounces and curved 50/50 mi...
Optinvent Sarl See through OK
Medium FOV
Large eyebox
Injection molded
Lumus Ltd See through OK
Medium FOV
Large eyebox
Al...
Word side
6 mirrorsWorld side
Lumus Ltd, DK32 (40deg FOV, 1.7mm)
and DK40 (25 deg FOV. 2.0mm)
(cascaded mirrors extractors...
Optinvent Sarl See through OK
Medium FOV
Large eyebox
Injection molded
Lumus Ltd See through OK
Medium FOV
Large eyebox
Al...
H. Mukawa et al. A full color eyewear
display using holographic planar
waveguides. Proc. SID 2008.
(Some) Technology Challenges
• Vergence-accommodation conflict (VAC)
• Vestibular-visual conflict (motion sickness)
• AR •...
Exciting Engineering Aspects of VR/AR
• CPU, GPU
• IPU, DPU?
• sensors & imaging
• computer vision
• scene understanding
•...
Stanford EE 267
Stanford Computational Imaging Group
Stanford Computational Imaging Lab
Light Field Displays
Time-of-Flight & NLOS Imaging
Computational
Microscopy
Image Optim...
O’Toole et al., CVPR 2017
ProcessedMeasured
ProcessedMeasured
Gordon Wetzstein
Computational Imaging Group
Stanford University
stanford.edu/~gordonwz
www.computationalimaging.org
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Computational Near-eye Displays with Focus Cues - SID 2017 Seminar
Próxima SlideShare
Cargando en…5
×

Computational Near-eye Displays with Focus Cues - SID 2017 Seminar

666 visualizaciones

Publicado el

SID 2017 seminar on next-generation VR & AR displays.

Publicado en: Ingeniería
  • Sé el primero en comentar

  • Sé el primero en recomendar esto

Computational Near-eye Displays with Focus Cues - SID 2017 Seminar

  1. 1. Computational Near-eye Displays with Focus Cues Gordon Wetzstein Stanford University SID Display Week 2017 www.computationalimaging.org
  2. 2. Personal Computer e.g. Commodore PET 1983 Laptop e.g. Apple MacBook Smartphone e.g. Google Pixel AR/VR e.g. Microsoft Hololens ???
  3. 3. A Brief History of Virtual Reality 1838 1968 2012-2017 Stereoscopes Wheatstone, Brewster, … VR & AR Ivan Sutherland VR explosion Oculus, Sony, HTC, MS, … Nintendo Virtual Boy 1995 VR 2.0
  4. 4. Ivan Sutherland’s HMD • optical see-through AR, including: • displays (2x 1” CRTs) • rendering • head tracking • interaction • model generation • computer graphics • human-computer interaction I. Sutherland “A head-mounted three-dimensional display”, Fall Joint Computer Conference 1968
  5. 5. Where we are now IFIXIT teardown
  6. 6. Tutorial Overview • conventional, fixed-focus near-eye displays • focus cues & the vergence-accommodation conflict • advanced optics for VR with focus cues: • adaptive and gaze-contingent focus displays • monovision • volumetric and multi-plane displays • near-eye light field displays • Maxwellian-type displays • AR displays
  7. 7. Magnified Display 1 d + 1 d' = 1 f d d’ f • big challenge: virtual image appears at fixed focal plane! • no focus cues
  8. 8. Duane, 1912 Nearestfocusdistance Age (years) 8 16 24 32 40 48 56 64 72 4D (25cm) 8D (12.5cm) 12D (8cm) Importance of Focus Cues Decreases with Age - Presbyopia 0D (∞cm) 16D (6cm)
  9. 9. 4D / 25cm Optical Infinity Normal vision Nearsighted/myopic Farsighted/Hyperopic Presbyopic Focal range (range of clear vision) Modified from Pamplona et al, Proc. of SIGGRAPH 2010 Nearsightedness & Farsightedness
  10. 10. Relative Importance of Depth Cues Cutting&Vishton,1995
  11. 11. The Vergence-Accommodation Conflict (VAC)
  12. 12. Real World: Vergence & Accommodation Match!
  13. 13. Current VR Displays: Vergence & Accommodation Mismatch
  14. 14. Accommodation and Retinal Blur
  15. 15. Blur Gradient Driven Accommodation
  16. 16. Blur Gradient Driven Accommodation
  17. 17. Blur Gradient Driven Accommodation
  18. 18. Blur Gradient Driven Accommodation
  19. 19. Blur Gradient Driven Accommodation
  20. 20. Blur Gradient Driven Accommodation
  21. 21. Top View Real World: Vergence & Accommodation Match!
  22. 22. Top View Stereo Displays Today (including HMDs): Vergence-Accommodation Mismatch! Screen
  23. 23. VR Displays with Focus Cues 1. Adaptive and Gaze-contingent Focus
  24. 24. Magnified Display Display Lens Fixed Focus 1 d + 1 d' = 1 f d d’ f
  25. 25. Adaptive Focus Magnified Display Display Lens 1 d + 1 d' = 1 f actuator  vary d’
  26. 26. Adaptive Focus Magnified Display Display Lens focus-tunable lens  vary f 1 d + 1 d' = 1 f
  27. 27. Adaptive Focus - History • M. Heilig “Sensorama”, 1962 (US Patent #3,050,870) • P. Mills, H. Fuchs, S. Pizer “High-Speed Interaction On A Vibrating-Mirror 3D Display”, SPIE 0507 1984 • S. Shiwa, K. Omura, F. Kishino “Proposal for a 3-D display with accommodative compensation: 3DDAC”, JSID 1996 • S. McQuaide, E. Seibel, J. Kelly, B. Schowengerdt, T. Furness “A retinal scanning display system that produces multiple focal planes with a deformable membrane mirror”, Displays 2003 • S. Liu, D. Cheng, H. Hua “An optical see-through head mounted display with addressable focal planes”, Proc. ISMAR 2008 manual focus adjustment Heilig 1962 automatic focus adjustment Mills 1984 deformabe mirrors & lenses McQuaide 2003, Liu 2008
  28. 28. Padmanaban et al., PNAS 2017
  29. 29. Padmanaban et al., PNAS 2017
  30. 30. Padmanaban et al., PNAS 2017
  31. 31. Padmanaban et al., PNAS 2017
  32. 32. Padmanaban et al., PNAS 2017
  33. 33. Padmanaban et al., PNAS 2017
  34. 34. Padmanaban et al., PNAS 2017
  35. 35. at ACM SIGGRAPH 2016 EyeNetra.com
  36. 36. at ACM SIGGRAPH 2016 participants of the study, 152 total EyeNetra.com
  37. 37. Participants - Prescription Padmanaban et al., PNAS 2017 n = 70, ages 21-64
  38. 38. vergence accommodation Conventional Stereo / VR Display
  39. 39. vergence accommodation Removing VAC with Adaptive Focus
  40. 40. Follow the target with your eyes 4D (0.25m) 0.5D (2m) Task
  41. 41. Stimulus Padmanaban et al., PNAS 2017 Accommodative Response RelativeDistance[D] Time [s]
  42. 42. Stimulus Accommodation n = 59, mean gain = 0.29 Padmanaban et al., PNAS 2017 Accommodative Response RelativeDistance[D] Time [s]
  43. 43. Stimulus Padmanaban et al., PNAS 2017 Accommodative Response RelativeDistance[D] Time [s]
  44. 44. Stimulus Accommodation Padmanaban et al., PNAS 2017 Accommodative Response RelativeDistance[D] Time [s] n = 24, mean gain = 0.77
  45. 45. Padmanaban et al., PNAS 2017 Do Presbyopes Benefit from Dynamic Focus? Gain Age
  46. 46. Padmanaban et al., PNAS 2017 Do Presbyopes Benefit from Dynamic Focus? Gain Age conventional
  47. 47. Padmanaban et al., PNAS 2017 Do Presbyopes Benefit from Dynamic Focus? Gain Age conventional dynamic
  48. 48. Gaze-contingent Focus • non-presbyopes: adaptive focus is like real world, but needs eye tracking! HMD lens micro display virtual image eye tracking Padmanaban et al., PNAS 2017
  49. 49. Gaze-contingent Focus Padmanaban et al., PNAS 2017
  50. 50. Gaze-contingent Focus Padmanaban et al., PNAS 2017
  51. 51. Gaze-contingent Focus Padmanaban et al., PNAS 2017
  52. 52. at ACM SIGGRAPH 2016
  53. 53. Gaze-contingent Focus – User Preference Padmanaban et al., PNAS 2017
  54. 54. VR Displays with Focus Cues 2. Monovision
  55. 55. Monovision VR Konrad et al., SIGCHI 2016; Johnson et al., Optics Express 2016; Padmanaban et al., PNAS 2017
  56. 56. Monovision VR Konrad et al., SIGCHI 2016; Johnson et al., Optics Express 2016; Padmanaban et al., PNAS 2017 • monovision did not drive accommodation more than conventional • visually comfortable for most; particularly uncomfortable for some users
  57. 57. VR Displays with Focus Cues 3. Multiplane Displays
  58. 58. Multiplane VR Displays • Rolland J, Krueger M, Goon A (2000) Multifocal planes head-mounted displays. Applied Optics 39 • Akeley K, Watt S, Girshick A, Banks M (2004) A stereo display prototype with multiple focal distances. ACM Trans. Graph. (SIGGRAPH) • Waldkirch M, Lukowicz P, Tröster G (2004) Multiple imaging technique for extending depth of focus in retinal displays. Optics Express • Schowengerdt B, Seibel E (2006) True 3-d scanned voxel displays using single or multiple light sources. JSID • Liu S, Cheng D, Hua H (2008) An optical see-through head mounted display with addressable focal planes in Proc. ISMAR • Love GD et al. (2009) High-speed switchable lens enables the development of a volumetric stereoscopic display. Optics Express • … many more ... idea introduced Rolland et al. 2000 benchtop prototype Akeley 2004 near-eye display prototype Liu 2008, Love 2009
  59. 59. Multiplane VR Displays • Rolland J, Krueger M, Goon A (2000) Multifocal planes head-mounted displays. Applied Optics 39 • Akeley K, Watt S, Girshick A, Banks M (2004) A stereo display prototype with multiple focal distances. ACM Trans. Graph. (SIGGRAPH) • Waldkirch M, Lukowicz P, Tröster G (2004) Multiple imaging technique for extending depth of focus in retinal displays. Optics Express • Schowengerdt B, Seibel E (2006) True 3-d scanned voxel displays using single or multiple light sources. JSID • Liu S, Cheng D, Hua H (2008) An optical see-through head mounted display with addressable focal planes in Proc. ISMAR • Love GD et al. (2009) High-speed switchable lens enables the development of a volumetric stereoscopic display. Optics Express • … many more ... idea introduced Rolland et al. 2000 benchtop prototype Akeley 2004 near-eye display prototype Liu 2008, Love 2009
  60. 60. VR Displays with Focus Cues 4. Light Field Displays
  61. 61. Light Field CamerasLight Field Stereoscope Huang et al., SIGGRAPH 2015
  62. 62. Backlight Thin Spacer & 2nd panel (6mm) Magnifying Lenses LCD Panel Light Field Stereoscope Huang et al., SIGGRAPH 2015
  63. 63. Near-eye Light Field Displays Idea: project multiple different perspectives into different parts of the pupil!
  64. 64. Target Light Field Input: 4D light field for each eye
  65. 65. Multiplicative Two-layer Modulation Input: 4D light field for each eye
  66. 66. Multiplicative Two-layer Modulation Input: 4D light field for each eye
  67. 67. Multiplicative Two-layer Modulation Input: 4D light field for each eye
  68. 68. Multiplicative Two-layer Modulation Reconstruction: for layer t1 Tensor Displays, Wetzstein et al. 2012 Input: 4D light field for each eye
  69. 69. Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope Light Field Stereoscope Huang et al., SIGGRAPH 2015
  70. 70. Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope Light Field Stereoscope Huang et al., SIGGRAPH 2015
  71. 71. Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope Light Field Stereoscope Huang et al., SIGGRAPH 2015
  72. 72. Traditional HMDs - No Focus Cues The Light Field HMD Stereoscope Light Field Stereoscope Huang et al., SIGGRAPH 2015
  73. 73. Tensor Displays Wetzstein et al., SIGGRAPH 2012
  74. 74. Vision-correcting Display iPod Touch prototypeprinted transparency Huang et al., SIGGRAPH 2014
  75. 75. prototype 300 dpi or higher Huang et al., SIGGRAPH 2014
  76. 76. VR Displays with Focus Cues 5. Maxwellian-type Displays
  77. 77. Maxwellian-type Near-eye Displays • eyebox of display is a pinhole  very large depth of field (no retinal blur cue) • exit pupil size of ≤0.5 mm  accommodation in open loop • pinholes are dim and reduce eyebox severely! (not practical)
  78. 78. Maxwellian-type Near-eye Displays
  79. 79. Accommodation-invariant Near-eye Displays Konradetal.,SIGGRAPH2017
  80. 80. Accommodation-invariant Near-eye Displays Konrad et al., SIGGRAPH 2017
  81. 81. Accommodation-invariant Near-eye Displays Konradetal.,SIGGRAPH2017
  82. 82. Accommodation-invariant Near-eye Displays
  83. 83. Accommodation-invariant Near-eye Displays Konrad et al., SIGGRAPH 2017
  84. 84. Other Upcoming Work at SIGGRAPH 2017 “Focal Surface Displays”, Matsuda et al. “Accommodation and Comfort in Head Mounted Displays”, Koulieris et al. “Holographic Near-eye Displays for Virtual and Augmented Reality”, Maimone et al.
  85. 85. Summary • focus cues in VR/AR are challenging • adaptive focus can correct for refractive errors (myopia, hyperopia) • gaze-contingent focus gives natural focus cues for non-presbyopes, but require eyes tracking • presbyopes require fixed focal plane with correction • multiplane displays require very high speed microdisplays • monovision has not demonstrated significant improvements • Maxwellian-type displays can be interesting, but provide small eyebox • light field displays may be the “ultimate” display  need to solve “diffraction problem”
  86. 86. Overview of Optical See-through AR Displays
  87. 87. slidebyBernardKress Consumer Applications – Prescription Glasses
  88. 88. Google Glass small!
  89. 89. Google Glass
  90. 90. Pepper’s Ghost 1862
  91. 91. Microsoft Hololens – Waveguide with Holographic Optical Elements
  92. 92. Oculus DK2 (115 deg) FOV (diag) and FOV location in some of the current products Sony Morpheus (90 deg) Sony HMZ-T3 (51deg) Lumus DK-32 (40 deg) Zeiss cinemizer (35 deg) Optinvent ORA (24 deg) Epson Moverio (23 deg) Occlusion display See through display Google Glass (15deg) Vuzix M-100 (16deg) Recon Jet (16deg) slidebyBernardKress
  93. 93. FOV and angular resolution for current productsAngularresolution(pixels/deg) FOV (diag, deg) Eye resolution limit (1.2 arc min or around 50 pixels/deg) 50 40 30 20 10 10 20 30 40 50 60 70 80 90 100 110 Glass Epson Moverio Vuzix m100 Oculus DK1 Sony HMZ T3 Optinvent Lumus Sony Morpheus Resolution measured in dpd rather than dpi Oculus DK2 slidebyBernardKress
  94. 94. There are two main optical HMD architectures slidebyBernardKress
  95. 95. Opaque Bird bath Free space Waveguide EPE Lightguide There are 5 main head-worn optical platform architectures developed in industry today See through architectures slidebyBernardKress
  96. 96. Oculus DK2 Sony HMZ Vuzix Occlusion Huge FOV Large eyebox Opaque Bird bath Free space Waveguide EPE Lightguide slidebyBernardKress
  97. 97. slidebyBernardKress
  98. 98. Oculus DK2 Sony HMZ Vuzix Vuzix M100 MyVu … Occlusion Huge FOV Large eyebox Partially occluded Small FOV Medium eye box Opaque Bird bath Free space Waveguide EPE Lightguide slidebyBernardKress
  99. 99. Most of the early occlusion HMDs are based on the combination of a transmission lens and a 45 degrees mirror (either TIR or coated) , in either monocular or binocular version. MicroOptical Corp. – MyVu Corp Recon Instruments Corp. slidebyBernardKress
  100. 100. “Bird bath” optical architectures Google Glass RockChip Ltd ITRI Good see through Small FOV Medium eye box Opaque Bird bath Free space Waveguide EPE Lightguide slidebyBernardKress
  101. 101. Google Glass slidebyBernardKress
  102. 102. Laster Sarl ODA Labs … Bug eye optics Large FOV Medium eyebox Temple projector Opaque Bird bath Free space Waveguide EPE Lightguide slidebyBernardKress
  103. 103. The notion of « Bug Eye » in HMD combiner opticsslidebyBernardKress
  104. 104. This is what we mean with the « bug-eye » concept slidebyBernardKress
  105. 105. ORA Associates (US) Reflective “bug eye” combiner Binocular version Laster Sarl (Fr) Reflective combiner (pupil forming architecture) slidebyBernardKress
  106. 106. Canon Ltd Motorola HC1 Kopin Golden eye Distorted see-through (or opaque) Medium FOV Medium eyebox Without complement piece Opaque Bird bath Free space Waveguide EPE Lightguide slidebyBernardKress
  107. 107. Canon Ltd Motorola HC1 Kopin Golden eye E-Magin Optics Fraunhofer Distorted see-through (or opaque) Medium FOV Medium eyebox Good see through Medium FOV Medium eye box Without complement piece With complement piece Opaque Bird bath Free space Waveguide EPE Lightguide slidebyBernardKress
  108. 108. TIR / free-form surfaces combiner (including eye tracking) Example with complement piece for see through slidebyBernardKress [Hua et al. 2013]
  109. 109. Epson Ltd Moverio 1 & 2 … OK see through Medium FOV Medium eyebox Curve coated reflector combiner Opaque Bird bath Free space Waveguide EPE Lightguide slidebyBernardKress
  110. 110. Epson Corp, Moverio (first and second generations) 3 collimation lenses, prism injector, 6 TIR bounces and curved 50/50 mirror Conventional optics (linear mirror in first gen and curved mirror in second gen) slidebyBernardKress
  111. 111. Optinvent Sarl See through OK Medium FOV Large eyebox Injection molded Lumus Ltd See through OK Medium FOV Large eyebox All glass various coatings Micro-prism combiner Cascaded coated mirrors combiner Opaque Bird bath Free space slidebyBernardKress Waveguide EPE Lightguide
  112. 112. Word side 6 mirrorsWorld side Lumus Ltd, DK32 (40deg FOV, 1.7mm) and DK40 (25 deg FOV. 2.0mm) (cascaded mirrors extractors) Uses side collimator and prism coupler slidebyBernardKress
  113. 113. Optinvent Sarl See through OK Medium FOV Large eyebox Injection molded Lumus Ltd See through OK Medium FOV Large eyebox All glass various coatings Sony Ltd Vuzix /Nokia (M2000AR) BAE Q-sight Good see through Medium FOV Large eyebox Photopolymer Micro-prism combiner Cascaded coated mirrors combiner Volume holographic combiner Diffractive combiner Opaque Bird bath Free space Waveguide EPE Lightguide slidebyBernardKress
  114. 114. H. Mukawa et al. A full color eyewear display using holographic planar waveguides. Proc. SID 2008.
  115. 115. (Some) Technology Challenges • Vergence-accommodation conflict (VAC) • Vestibular-visual conflict (motion sickness) • AR • occlusions • aesthetics / form factor • battery life • heat • wireless operation • low-power computer vision • registration of physical / virtual world and eyes • consistent lighting • scanning real world • VAC more important • display contrast & brightness • fast, embedded GPUs • …
  116. 116. Exciting Engineering Aspects of VR/AR • CPU, GPU • IPU, DPU? • sensors & imaging • computer vision • scene understanding • human perception • displays: visual, auditory, vestibular, haptic, …• VR cameras • cloud computing • shared experiences • HCI • compression, streaming
  117. 117. Stanford EE 267
  118. 118. Stanford Computational Imaging Group
  119. 119. Stanford Computational Imaging Lab Light Field Displays Time-of-Flight & NLOS Imaging Computational Microscopy Image Optimization Light Field Cameras
  120. 120. O’Toole et al., CVPR 2017
  121. 121. ProcessedMeasured
  122. 122. ProcessedMeasured
  123. 123. Gordon Wetzstein Computational Imaging Group Stanford University stanford.edu/~gordonwz www.computationalimaging.org

×