2. Light Field Inside a Camera
Lenslet-based Light Field camera
[Adelson and Wang, 1992, Ng et al. 2005 ]
3. Stanford Plenoptic Camera
[Ng et al 2005]
Contax medium format camera
Kodak 16-megapixel sensor
Adaptive Optics microlens array
125μ square-sided microlenses
4000
4000 pixels
292
292 lenses = 14
14 pixels per lens
21. Computing 4D Light Field
2D Sensor Photo, 1800*1800
2D Fourier Transform
2D
FFT
9*9=81 spectral copies
Rearrange 2D tiles into 4D planes
4D IFFT
4D Light Field
200*200*9*9
200*200*9*9
22. Mask = more information?
Mask?
Sensor
Sensor
Mask
Digital Refocusing
Sensor
Mask
Heterodyne Light
Field Camera
[Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan], Sig
graph 2007
23. MERL
Veeraraghavan, Raskar, Agrawal,
Mohan & Tumblin
Mask-Enhanced Cameras: Heterodyned Light Fields & Coded Aperture
Differences with Plenoptic Camera
Sensor
Sensor
Microlens
array
Plenoptic Camera
Mask
Heterodyne Camera
• Micro-lens array
• Narrowband Cosine Mask
• Samples individual rays
• Samples coded comb of rays
• Needs alignment precision
• More flexible
• Some pixels wasted
• No wastage
- Half brightness, diffraction
24. Novel Sensors
• Color
– Foveon
• Dynamic Range
– HDR Camera, Log sensing
– Gradient sensing
• Identity
– Demodulation
• 3D
– ZCam, Canesta
• Motion
– Line scan Camera
– Flutter Shutter
26. High Dynamic Range
Fuji's SuperCCD S3 Pro
Sensor with high and low sensitivity sensors
per pixel location
27. Gradient Camera
• Sense Pixel Intensity Difference with
unknown locally adaptive gain
• Reconstruct image from 2D gradient field
Ramesh Raskar, MERL
Work with Jack Tumblin, Northwestern U,
Amit Agrawal, U of Maryland
28. High Dynamic Range Images
Scene
Intensity camera
saturation map
Gradient camera
saturation map
Intensity camera fail to capture range
Gradients saturate at very few isolated pixels
44. Future ..
• „Cloth-cam‟: „Wallpaper-cam‟
– Fusion of 4D light emission and 4D capture in the surface of a cloth…
• Human Augmentation
– Cameras to replace human eyes, for blind or limited vision
– Camera on the „back‟
• More Sensors
– GPS, Compass, Temperature, fingerprint recognition, face recognition
– When, Where, What, How .. Why?
• Photo Sharing and Community:
– Photo Clip and Scene Completion
– City Scanning, Live