Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (17)
Similar a SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)
Similar a SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3) (20)
Último
Último (20)
SIGGRAPH 2014 Course on Computational Cameras and Displays (part 3)
- 1. Compressive Display Systems Gordon Wetzstein MIT / Stanford University media.mit.edu/~gordonw displayblocks.org
- 3. This slide has a 16:9 media window Evolution? 1928 2014
- 5. uberpixel
- 6. Next-generation Devices computation optics & electronics human visual system interaction optics (compressive) computationsensing Computational & Compressive Displays Computational Imaging
- 8. Images: Wikipedia, Shinya Yoshioka
- 9. This slide has a 16:9 media window Nature Image: Desafio Monteverde and Arenal Volcano Tours Costa Rica
- 11. This slide has a 16:9 media window Compressive Light Field Displays
- 12. This slide has a 16:9 media window
- 13. This slide has a 16:9 media window viewer moves right viewermovesdown 4D Light Field
- 16. 2D display barrier Parallax Barriers – Ives 1903 • low resolution & very dim
- 17. Integral Imaging – Lippmann 1908 • low-res, but brighter than parallax barriers 2D display lenslets
- 18. patent drawings - early 20th century
- 19. multiplexing nonlinear compression optimization user experience content community next M N O C U C C N Structural Formula for Compressive Displays
- 20. multiplexing nonlinear compression optimization user experience content community next M N O C U C C N Nonlinear Pixel Interaction & Coupling
- 21. layer 2 layer 1 Conventional Parallax Barriers
- 22. layer 2 layer 1 Conventional Parallax Barriers blocked! p1 p2 l = p1*p2 nonlinear
- 23. layer 2 layer 1 Most Volumetric Displays / Additive Layers e.g., LEDs or transparent OLEDs not blocked! p1 p2 l = p1+p2 linear
- 24. Holography – Nonlinear Interaction plane wave hologram emitted wavefront: screen or retina received intensity: I(x) = T U(x){ } 2 = Re T U(x){ }{ } 2 +Im T U(x){ }{ } 2 U(x)
- 25. Holography – Coupling plane wave hologram W(x,u = lsin(q)) = t x + x' 2 æ è ç ö ø ÷t x - x' 2 æ è ç ö ø ÷e2pix'u dx'ò Fourier transform of all points interacting with each other! nonlinear interaction & pixel coupling!
- 26. attenuation layers with spacers backlight Layered 3D – SIGGRAPH 2011
- 27. Layered 3D – SIGGRAPH 2011 layer 2 layer K layer 1 …
- 28. Layered 3D – SIGGRAPH 2011 layer 2 layer K layer 1 … p1 p2 pK l = p1*p2*…*pK
- 29. Layered 3D – SIGGRAPH 2011 … layer 2 layer K layer 1
- 30. Tensor Displays – SIGGRAPH 2012 directional backlight three layer Reconfigurable
- 32. time Tensor Displays – Directional Backlight Perceptual Integration layer 2 microlens array layer 1
- 33. LCD with directional backlight, rank 6 LCD + directional BL conventional lenslets view from above
- 34. LCD with directional backlight, rank 6 (as seen by observer)
- 35. front LCD directional backlight Filmed with High-speed Camera
- 36. four stacked liquid crystal panels two crossed polarizers
- 37. LCD 1 LCD 2 LCD 3 backlight Polarization Fields – SIGGRAPH ASIA 2011 polarizer polarizer color filter array polarizer color filter array polarizer color filter array
- 38. backlight Polarization Fields – SIGGRAPH ASIA 2011 LCD 1 LCD 2 LCD 3 polarizer polarizer
- 39. image formation f1 f2 f3 L(x,q)=sin2 Q(x,q)( ) backlight Polarization Fields – SIGGRAPH ASIA 2011 LCD 1 LCD 2 LCD 3
- 41. Which Light Field is More Compressible? a b
- 42. Which Light Field is More Compressible? a b
- 43. This slide has a 16:9 media window 4D Light Field Uniform or Directional Backlight Stacked Layers Nonnegative Tensor Factorization Display-adaptive Compression Compressive Computed Tomography (LCDs or Transparencies) Optics Observer = Decoder
- 44. Give those Pixels a Break! Applied Mathematics • sparse optimization • low-rank factorization • computed tomography • … Benefits for Optics & Electronics • fewer pixels • relaxation on refresh rate • thinner form factors • …
- 45. Parallax Barriers 1903 Time-Shifted Parallax Barriers 2007 t HR3D SIG Asia 2010 t t Layered 3D SIGGRAPH 2011 Tensor Displays SIGGRAPH 2012 Conventional Parallax Barriers layer 2 layer 1 From Conventional to Compressive Displays
- 46. … … Parallax Barriers 1903 Time-Shifted Parallax Barriers 2007 t HR3D SIG Asia 2010 t t Layered 3D SIGGRAPH 2011 Tensor Displays SIGGRAPH 2012 … time Perceptual Integration Tensor Displays – Multilayer & Directional Backlighting From Conventional to Compressive Displays
- 47. Parallax Barriers 1903 Time-Shifted Parallax Barriers 2007 t HR3D SIG Asia 2010 t t Layered 3D SIGGRAPH 2011 Tensor Displays SIGGRAPH 2012 Perceptual Integration Tensor Displays – Directional Backlighting time thin! From Conventional to Compressive Displays
- 48. multiplexing nonlinear compression optimization user experience content community next M N O C U C C N Real-time Optimization
- 49. This slide has a 16:9 media window 4D Light Field Display-adaptive Compression Compressive Optics via optimization
- 50. attenuation layers with spacers backlight Layered 3D – SIGGRAPH 2011
- 51. ? • limited baseline tomography • use algebraic approaches! Layered 3D – SIGGRAPH 2011
- 52. This slide has a 16:9 media window Computed Tomography (CT) Image:Wikipedia x-ray source x-ray sensor 3D Reconstruction Reconstructed 2D Slices
- 53. Tomographic Light Field Synthesis q 2D light field x x q backlight attenuation volume virtual planes image formation L(x,q) = e - m(r)dr c ò log L x,q( )( )=- m(r)dr c ò 2 2 0 P)log(argmin L tomographic synthesis
- 54. Limits of Tomographic Light Field Synthesis image formation L(x,q) = e - m(r)dr c ò log L x,q( )( )=- m(r)dr c ò 2 2 0 P)log(argmin L tomographic synthesis log space … p1 p2 pK … p1 p2 pK time l = p1*…*pK … p1 p2 pK log(l) = log(p1)+…+log(pK) l = (p1*p2*…*pK)1+…+(p1*p2*…*pK)N loglin ???
- 55. backlight rear layer front layer two-layer light field display Low-rank Light Field Factorization light field fm (1)(x1) fm (2)(x2) x1 x2 L(x1, x2)
- 56. backlight rear layer front layer two-layer light field display fm (1)(x1) fm (2)(x2) x1 x2 L(x1, x2) ` front layer rearlayer rank-1 Lanman et al. – SIGGRAPH Asia 2010 Low-rank Light Field Factorization
- 57. backlight rear layer front layer two-layer light field display fm (1)(x1) fm (2)(x2) x1 x2 L(x1, x2) ` rank-4 Lanman et al. – SIGGRAPH Asia 2010 high-speed LCDs = perceptual average Low-rank Light Field Factorization
- 58. ` rank-4 F G L ~ Lanman et al. – SIGGRAPH Asia 2010 Low-rank Light Field Factorization high-speed LCDs = perceptual average arg min F,G L - FG W 2 , for F,G ³ 0 objective function:
- 59. fm (1)(x1) fm (2)(x2) fm (3)(x3) x1 x2 x3 L(x2, x3) multi-layer light field display backlight rear layer middle layer front layer light field Light Field Slice Representation
- 60. fm (1)(x1) fm (2)(x2) fm (3)(x3) x1 x2 x3 L(x1,x2,x3) RearLayer x3 x2 x1 L(x1,x2,x3) light field tensor backlight rear layer middle layer front layer light field multi-layer light field display Light Field Tensor Representation
- 61. fm (1)(x1) fm (2)(x2) fm (3)(x3) x1 x2 x3 RearLayer x3 x2 x1 L(x1,x2,x3) L(x1,x2,x3) light field tensor backlight rear layer middle layer front layer light field multi-layer light field display Light Field Tensor Representation
- 62. fm (1)(x1) fm (2)(x2) fm (3)(x3) RearLayer light field tensor backlight rear layer middle layer front layer light field multi-layer light field display Light Field Tensor Representation
- 63. fm (1)(x1) fm (2)(x2) fm (3)(x3) RearLayer light field tensor backlight rear layer middle layer front layer light field multi-layer light field display Light Field Tensor Representation
- 64. fm (1)(x1) fm (2)(x2) fm (3)(x3) RearLayer light field tensor backlight rear layer middle layer front layer light field multi-layer light field display Light Field Tensor Representation
- 65. frame M + ... + Target Light Field Tensor frame 1 nonnegative tensor factorization (NTF) Rank-M Approximation perceptual integration frame 2 + Low-rank Tensor Factorization
- 66. iterative update rules nonlinear (multilinear) optimization problem Low-rank Tensor Factorization • standard form – Tensor Compendium • multiplicative update keep factors positive • basically steepest descent with fixed step length
- 67. “standard” formulationalternate formulation Low-rank Tensor Factorization
- 68. alternate formulation forward projection (multiview rendering) back projection (projective texture mapping) Efficient GPU Implementation Tensor Factorization - Implementation
- 71. Vision-correcting Display perceived image displayed image SIGGRAPH 2014 - Display Session, Tue morning
- 72. Vision-correcting Display iPod Touch prototypeprinted transparency SIGGRAPH 2014 - Display Session, Tue morning
- 73. prototype construction 300 dpi or higher
- 77. Compressive Multi-mode Display Optics Express 2014
- 78. This slide has a 16:9 media window High-speed LCD High-speed LCD Electronically-switchable Diffuser
- 79. High-speed LCD High-speed LCD Electronically-switchable Diffuser OFF 3D Display Mode
- 80. Light Field Factorization – LCD Patterns Front Layer Rear Layer
- 81. Light Field Factorization – Results
- 82. High Dynamic Range Display Mode High-speed LCD High-speed LCD Electronically-switchable Diffuser OFF
- 83. Target HDR Image
- 84. Solver = Light Field without Parallax Front Layer Rear Layer
- 86. Superresolution Display Mode High-speed LCD High-speed LCD Electronically-switchable Diffuser ON
- 89. Light Field Projection SIGGRAPH 2014 – ETech & Paper
- 90. Compressive Light Field Projection
- 93. multiplexing nonlinear compression optimization user experience content community next N O C U C C N Structural Formula for Compressive Displays M Mobile Displays Optics Express 2014 Projection Displays SIGGRAPH 2014 Coded Illuminationfor Microscopy & Lithography Head Mounted Displays CGF 2010, SIGGRAPH 2014 Monitors / TVs SIG2011,2012,2013, SIGAsia 2009,2011
- 94. Cool Displays at SIGGRAPH 2014 Technical Papers Sessions Emerging Technologies • Displays, Tuesday 10:45-12:15, Hall A • Computational Sensing and Display, Tue 3:45-5:15, Hall A • AR & VR Displays • Light Field Projection, HDR Projection • much more!
- 95. Gordon Wetzstein Massachusetts Institute of Technology media.mit.edu/~gordonw displayblocks.org Matt Hirsch (MIT) Doug Lanman (MIT/NVIDIA/Oculus VR) Andrew Maimone (UNC) Felix Heide (UBC) Fu-Chung Huang (UC Berkeley) Belen Masia (University of Zaragoza) collaborators sources of funding & hardware Wolfgang Heidrich (UBC/KAUST) Ramesh Raskar (MIT) Diego Gutierrez (University of Zaragoza) Brian Barsky (UC Berkeley) Henry Fuchs (UNC)
- 96. Gordon Wetzstein Massachusetts Institute of Technology media.mit.edu/~gordonw displayblocks.org Stanford looking for: • students, postdocs, interns • (industry) collaborators