Raskar Coded Opto Charlotte
•Descargar como PPT, PDF•
2 recomendaciones•874 vistas
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Similar a Raskar Coded Opto Charlotte
Similar a Raskar Coded Opto Charlotte (20)
Más de Camera Culture Group, MIT Media Lab
Más de Camera Culture Group, MIT Media Lab (20)
Último
Último (20)
Raskar Coded Opto Charlotte
- 1. Mitsubishi Electric Research Laboratories Raskar 2007 Media Lab, MIT Cambridge, MA Less is More:Less is More: Coded Computational PhotographyCoded Computational Photography Ramesh Raskar P r o j e c t o r T a g s P o s = 0 P o s = 2 5 5
- 2. MIT Media Lab Camera Culture Ramesh Raskar Camera Culture
- 5. Short Exposure Traditional MURA Deblurred Result Captured Single Photo Shutter Banding Artifacts and some spatial frequencies are lost Dark and noisy
- 6. Blurring == Convolution Traditional Camera: Shutter is OPEN: Box Filter PSF == Sinc Function ω Sharp Photo Blurred Photo Fourier Transform
- 7. Flutter Shutter: Shutter is OPEN and CLOSED Preserves High Spatial Frequencies Sharp Photo Blurred Photo PSF == Broadband Function Fourier Transform
- 8. Flutter Shutter CameraFlutter Shutter Camera Raskar, Agrawal, Tumblin [Siggraph2006] LCD opacity switched in coded sequence
- 11. Coded Exposure Temporal 1-D broadband code: Motion Deblurring Coded Aperture Spatial 2-D broadband mask: Focus Deblurring
- 12. Coded Aperture CameraCoded Aperture Camera The aperture of a 100 mm lens is modified Rest of the camera is unmodified Insert a coded mask with chosen binary pattern
- 14. Out of Focus Photo: Open Aperture
- 15. Out of Focus Photo: Coded Aperture
- 18. Less is MoreLess is More Blocking Light == More InformationBlocking Light == More Information Coding in TimeCoding in Time Coding in SpaceCoding in Space
- 19. Less is More ..Less is More .. • Coded Exposure – Motion Deblurring [2006] • Coded Aperture – Focus Deblurring [2007] • Optical Heterodyning – Light Field Capture [2007] • Coded Spectrum – Agile Wavelength Profile [2008] • Coded Illumination – Motion Capture [2007] – Multi-flash: Shape Contours [2004] P r o je c t o r T a g s P o s = 0 P o s = 2 5 5
- 20. Computational PhotographyComputational Photography 1. Epsilon Photography – Low-level vision: Pixels – Multi-photos by perturbing camera parameters – HDR, panorama, … – ‘Ultimate camera’ 1. Coded Photography – Single/few snapshot – Reversible encoding of data – Additional sensors/optics/illum – ‘Scene analysis’ : (Consumer software?) 1. Essence Photography – Beyond single view/illum – Not mimic human eye – ‘New art form’
- 21. Computational PhotographyComputational Photography 1. Epsilon Photography – Low-level Vision: Pixels – Multiphotos by perturbing camera parameters – HDR, panorama – ‘Ultimate camera’ 1. Coded Photography – Mid-Level Cues: • Regions, Edges, Motion, Direct/global – Single/few snapshot • Reversible encoding of data – Additional sensors/optics/illum – ‘Scene analysis’ 1. Essence Photography – Not mimic human eye – Beyond single view/illum – ‘New artform’
- 22. Computational PhotographyComputational Photography 1. Epsilon Photography – Multiphotos by varying camera parameters – HDR, panorama – ‘Ultimate camera’: (Photo-editor) 1. Coded Photography – Single/few snapshot – Reversible encoding of data – Additional sensors/optics/illum – ‘Scene analysis’ : (Next software?) 1. Essence Photography – High-level understanding • Not mimic human eye • Beyond single view/illum – ‘New artform’
- 23. Mask? Sensor Mask SensorMask? Sensor Mask Sensor Mask? Sensor 4D Light Field from 2D Photo: Heterodyne Light Field Camera Full Resolution Digital Refocusing: Coded Aperture Camera
- 24. Light Field Inside a CameraLight Field Inside a Camera
- 25. Lenslet-based Light Field cameraLenslet-based Light Field camera [Adelson and Wang, 1992, Ng et al. 2005 ] Light Field Inside a CameraLight Field Inside a Camera
- 26. Stanford Plenoptic CameraStanford Plenoptic Camera [Ng et al 2005][Ng et al 2005] 4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens Contax medium format camera Kodak 16-megapixel sensor Adaptive Optics microlens array 125μ square-sided microlenses
- 27. Digital RefocusingDigital Refocusing [Ng et al 2005][Ng et al 2005] Can we achieve this with aCan we achieve this with a MaskMask alone?alone?
- 28. Mask based Light Field Camera Mask Sensor [Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ]
- 29. How to Capture 4D Light Field with 2D Sensor ? What should be the pattern of the mask ?
- 30. Radio Frequency HeterodyningRadio Frequency Heterodyning Baseband Audio Signal Receiver: DemodulationHigh Freq Carrier 100 MHz Reference Carrier Incoming Signal 99 MHz
- 31. Optical HeterodyningOptical Heterodyning Photographic Signal (Light Field) Carrier Incident Modulated Signal Reference Carrier Main LensObject Mask Sensor Recovered Light Field Software Demodulation Baseband Audio Signal Receiver: DemodulationHigh Freq Carrier 100 MHz Reference Carrier Incoming Signal 99 MHz
- 32. Captured 2D Photo Encoding due to Mask
- 33. 2D FFT Traditional Camera Photo Heterodyne Camera Photo Magnitude of 2D FFT 2D FFT Magnitude of 2D FFT
- 34. Computing 4D Light Field 2D Sensor Photo, 1800*1800 2D Fourier Transform, 1800*1800 2D FFT Rearrange 2D tiles into 4D planes 200*200*9*94D IFFT 4D Light Field 9*9=81 spectral copies 200*200*9*9
- 35. A Theory of Mask-Enhanced CameraA Theory of Mask-Enhanced Camera Main LensObject Mask Sensor •Mask == Light Field Modulator •Intensity of ray gets multiplied by Mask •Convolution in Frequency domain
- 36. fθ fx fθ0 fx0 Band-limited Light Field Sensor Slice – Fourier Slice Theorem Photo = Slice of Light Field in Fourier Domain [Ren Ng, SIGGRAPH 2005]
- 37. How to Capture 2D Light Field with 1D Sensor ? fθ fx fθ0 fx0 Band-limited Light Field Sensor Slice Fourier Light Field Space
- 38. Extra sensor bandwidth cannot capture extra dimension of the light field fθ fx fθ0 fx0 Sensor Slice Extra sensor bandwidth
- 40. Solution: Modulation Theorem Make spectral copies of 2D light field fθ fx fθ0 fx0 Modulation Function
- 41. fθ Modulated Light Field fx fθ0 fx0 Modulation Function Sensor Slice captures entire Light Field
- 42. Demodulation to recover Light Field fθ fx Reshape 1D Fourier Transform into 2D 1D Fourier Transform of Sensor Signal
- 43. fθ fx fθ0 fx0 Modulation Function == Sum of Impulses Physical Mask = Sum of Cosines
- 44. 1/f0 Mask Tile Cosine Mask Used
- 45. Where to place the Mask? Mask Sensor Mask Sensor Mask Modulation Function Mask Modulation Function fx fθ
- 46. Mask Sensor Where to place the Mask? Mask Modulation Functionfx fθ
- 47. Captured 2D Photo Encoding due to Cosine Mask
- 48. Computing 4D Light Field 2D Sensor Photo, 1800*1800 2D Fourier Transform 2D FFT Rearrange 2D tiles into 4D planes 200*200*9*94D IFFT 4D Light Field 9*9=81 spectral copies 200*200*9*9
- 49. Digital Refocusing Only cone in focus Captured Photo
- 50. Full resolution 2D image of Focused Scene Parts Captured 2D Photo Image of White Lambertian Plane divide
- 51. Coding and Modulation in Camera Using MasksCoding and Modulation in Camera Using Masks Mask? Sensor Mask Sensor Mask Sensor Coded Aperture for Full Resolution Digital Refocusing Heterodyne Light Field Camera
- 52. Agile Spectrum Imaging With Ankit Mohan, Jack Tumblin [Eurographics 2008]
- 53. C B A A’ B’ C’ Pinhole Lens L1 Prism or Diffraction Grating Lens L2 Sensor Rainbow Plane C’ ’ B’ ’ A’ ’ Scene Rainbow Plane inside Camera
- 59. Glare Reduction/Enhancement usingGlare Reduction/Enhancement using 4D Ray Sampling4D Ray Sampling Captured Glare Reduced Glare Enhanced
- 60. i j x Sensor u Glare = low frequency noise in 2D •But is high frequency noise in 4D •Remove via simple outlier rejection
- 62. Mask based Light Field Camera Mask Sensor
- 69. Glare Reduction/Enhancement usingGlare Reduction/Enhancement using 4D Ray Sampling4D Ray Sampling [Siggraph 2008][Siggraph 2008] Captured Glare Reduced Glare Enhanced
- 70. • 6D Display • Multi-flash Camera • Computational Photography – Online Siggraph Course Notes – Google: ‘Computational Photography’ Other ProjectsOther Projects
- 71. Coded Computational Photography • Coded Exposure – Motion Deblurring [2006] • Coded Aperture – Focus Deblurring [2007] – Glare reduction [2008] • Optical Heterodyning – Light Field Capture [2007] • Coded Illumination – Motion Capture [2007] – Multi-flash: Shape Contours [2004] • Coded Spectrum – Agile Wavelength Profile [2008] • Epsilon->Coded->Essence Photography http://raskar.info
- 73. Blind CameraBlind Camera Sascha Pohflepp, U of the Art, Berlin, 2006
- 74. Coded Computational Photography • Coded Exposure – Motion Deblurring [2006] • Coded Aperture – Focus Deblurring [2007] – Glare reduction [2008] • Optical Heterodyning – Light Field Capture [2007] • Coded Illumination – Motion Capture [2007] – Multi-flash: Shape Contours [2004] • Coded Spectrum – Agile Wavelength Profile [2008] • Epsilon->Coded->Essence Photography http://raskar.info
Notas del editor
- “Photography is the art of exclusion” Rockwell
- Car result
- Low signal to noise ratio
- How is the blurred image formed? It’s a convolution with box filter
- Coded exposure makes the filter broadband
- Comparisons
- License plate example: Blur = 60 pixels Can you guess what the car make is ? How many think it is the Audi ? Actually it is a Folksvagon.
- Cannot handle edges in blur
- 2D result
- Reversibly encode all the information in this otherwise blurred photo
- The glint out of focus shows the unusual pattern.
- Difficult to argue that the worm is performing high quality deconvolution to form an image. But in our group we are setting up experiments by creating active lighting probe to understand how worms perform visual analysis.
- Talk about limitations: Colocated artifacts, color coherency, ref can’t be obtain by subtraction
- Stereo-pair is a simple example of coded photography. Many decomposition problems, direct/global, diffuse/specular,
- Inference and perception are important. Intent and goal of the photo is important. The same way camera put photorealistic art out of business, maybe this new artform will put the traditional camera out of business. Because we wont really care about a photo, merely a recording of light but a form that captures meaningful subset of the visual experience. Multiperspective photos. Photosynth is an example.
- Maybe all the consumer photographer wants is a black box with big red button. No optics, sensors or flash. If I am standing the middle of times square and I need to take a photo. Do I really need a fancy camera?
- The camera can trawl on flickr and retrieve a photo that is roughly taken at the same position, at the same time of day. Maybe all the consumer wants is a blind camera.