We propose a novel camera setup in which both the lens and the sensor are perturbed during the exposure. We analyze the defocus effects produced by such a setup, and use it to demonstrate new methods for simulating a lens with a larger effective aperture size (i.e., shallower depth of field) and methods for achieving approximately depth-independent defocus blur size. We achieve exaggerated, programmable, and pleasing bokeh with relatively small aperture sizes such as those found on cell phone cameras. Destabilizing the standard alignment of the sensor and lens allows us to introduce programmable defocus effects and achieve greater flexibility in the image capture process.
Understanding Discord NSFW Servers A Guide for Responsible Users.pdf
Image Destabilization ICCP 2009
1. MIT Media Lab Camera Culture
Image Destabilization:
Programmable Defocus using
Lens and Sensor Motion
Ankit Mohan, Douglas Lanman,
Shinsaku Hiura, Ramesh Raskar
MIT Media Lab
2. MIT Media Lab Camera Culture
Defocus Blur
Lots of glass; Heavy; Bulky; Expensive
3. MIT Media Lab Camera Culture
Image Destabilization
Camera
Lens Sensor
Static
Scene
4. MIT Media Lab Camera Culture
Image Destabilization
Camera
Static
Scene
Lens Motion Sensor Motion
5. MIT Media Lab Camera Culture
f8
Related Work
f/4
f/2
[Bae and Durand 2007] extrapolated
aperture
f/1
[Hasinoff and Kutulakos 2007]
[Vaish et al. 2004]
[Hiura et al. 2009]
6. MIT Media Lab Camera Culture
Laminography
Motion direction
X-Ray Source
Plane of focus
X-Ray Sensor
Motion direction
Related technique: Time Delay and Integration (TDI)
7. MIT Media Lab Camera Culture
Lens based Focusing
Lens Sensor
A B’
A’
B
8. MIT Media Lab Camera Culture
Lens based Focusing
Lens Sensor
A B’
A’
B
9. MIT Media Lab Camera Culture
Smaller aperture Smaller defocus blur
Lens Sensor
A B’
A’
B
10. MIT Media Lab Camera Culture
Pinhole: All In-Focus
Pinhole Sensor
A B’
A’
B
11. MIT Media Lab Camera Culture
Shifting Pinhole
Pinhole Sensor
vp
A
B’
B
A’
12. MIT Media Lab Camera Culture
Shifting Pinhole
Pinhole Sensor
vp
A
B’
B A’
13. MIT Media Lab Camera Culture
Shifting Pinhole
Pinhole Sensor
vp
B’
A
A’
B
14. MIT Media Lab Camera Culture
Shifting Pinhole
Pinhole Sensor
vp
B’
A
A’
B
15. MIT Media Lab Camera Culture
Shifting Pinhole
Pinhole Sensor
vp
B’
A
tp
A’
B
da
db ds
16. MIT Media Lab Camera Culture
Shifting Pinhole and Sensor
Pinhole Sensor
vp vs
A
B’
B
A’
da
db ds
Focus Here
17. MIT Media Lab Camera Culture
Shifting Pinhole and Sensor
Pinhole Sensor
vp vs
B’
A
A’
B
da
db ds
Focus Here
18. MIT Media Lab Camera Culture
Shifting Pinhole and Sensor
Pinhole Sensor
B’
vp vs
A
A’
B
da
db ds
Focus Here
19. MIT Media Lab Camera Culture
Shifting Pinhole and Sensor
Pinhole Sensor
B’
vp vs
A
A’
B
da
db ds
Focus Here
20. MIT Media Lab Camera Culture
A Lens in Time!
Lens Equation:
Virtual Focal Length:
Virtual F-Number:
Analogous to shift and sum based
Light field re-focusing.
22. MIT Media Lab Camera Culture
Adjusting the Focus Plane
all-in-focus pinhole image
23. MIT Media Lab Camera Culture
Adjusting the Focus Plane
focused in the front using destabilization (10 second exposure)
24. MIT Media Lab Camera Culture
Adjusting the Focus Plane
focused in the middle using destabilization (5 second exposure)
25. MIT Media Lab Camera Culture
Adjusting the Focus Plane
focused in the back using destabilization (10 second exposure)
26. MIT Media Lab Camera Culture
Adjusting the Virtual Aperture
focused in the middle using destabilization (5mm pinhole translation)
27. MIT Media Lab Camera Culture
Adjusting the Virtual Aperture
focused in the middle using destabilization (30mm pinhole translation)
28. MIT Media Lab Camera Culture
Shifting Lens and Sensor Defocus
Defocus Exaggeration
• Physical vs. synthetic focus focus
Similar physical and synthetic
real focus
virtual focus
aperture
sensor
29. MIT Media Lab Camera Culture
Defocus Exaggeration
static lens with an f/2.8 aperture
30. MIT Media Lab Camera Culture
Defocus Exaggeration
destabilization simulates a reduced f-number
31. MIT Media Lab Camera Culture
Defocus Invariance
• Differing physical and synthetic focus
real focus
virtual focus
aperture
sensor
32. MIT Media Lab Camera Culture
Defocus Invariance
• Related work
– [Nagahara et al. 2008]
– [Cathey and Dowski 1995]
[Nagahara et al. 2008] – [Levin et al. 2008]
• PSF not depth invariant
* = – only size is depth invariant
real PSF virtual PSF overall PSF
• Gaussian special case
– depth invariant PSF
* = – inversion is ill-conditioned
real PSF virtual PSF overall PSF
33. MIT Media Lab Camera Culture
Defocus Invariance
depth-invariant blur size (horizontal slit + destabilization)
34. MIT Media Lab Camera Culture
Defocus Invariance
Richardson-Lucy deconvolution result
35. MIT Media Lab Camera Culture
Tilted Sensor
focus plane aperture plane sensor plane
dC
C
D′
D
C′
dD
d′D
d′C
Scheimpflug intersection
36. MIT Media Lab Camera Culture
Tilted Sensor
focus plane aperture plane sensor plane
dC
C vp vs?
D
dD
37. MIT Media Lab Camera Culture
Tilted Sensor
focus plane aperture plane sensor plane
dC
C vp vs
D
dD
α
α
d′C
d′D
D′ C′
38. MIT Media Lab Camera Culture
Tuning the PSF
real focus
aperture
sensor
pinhole image (static f/22 aperture)
39. MIT Media Lab Camera Culture
Tuning the PSF
real focus
aperture
sensor
large aperture image (static f/2.8 aperture)
40. MIT Media Lab Camera Culture
Tuning the PSF
virtual focus
aperture
sensor
destabilized image using a pinhole (translated f/22 aperture)
41. MIT Media Lab Camera Culture
Tuning the PSF
real focus
virtual focus
aperture
sensor
destabilized image using a large aperture (translated f/2.8 aperture)
42. MIT Media Lab Camera Culture
Tuning the PSF
real focus
virtual focus
aperture
sensor
simulated aspheric lens using a vertical slit aperture and destabilization
43. MIT Media Lab Camera Culture
Extension to 2D Displacements
linear circular elliptical
“figure 8” hypocycloidal trispiral
44. MIT Media Lab Camera Culture
Large apertures with tiny lenses?
Benefits Limitations
• No time or light inefficiency • Coordinated mechanical
wrt cheap cameras movement required
• Exploits unused area around • Diffraction (due to small aperture)
the lens cannot be eliminated
• Compact design [Zhang and Levoy, tomorrow]
[Our group: augmented LF for wave analysis]
• With near-pinhole apertures
(mobile phones) many • Scene motion during exposure
possibilities
45. MIT Media Lab Camera Culture
Acknowledgements
Grace Woo Quinn Smithwick Gabriel Taubin Jaewon Kim
MIT CSAIL MIT Media Lab Brown University MIT Media Lab
MIT Media Lab: Camera Culture
46. MIT Media Lab Camera Culture
SLRs with tiny lenses?
• Analysis of space of relative lens/sensor displacement
• Destabilization as virtual focusing mechanism
• Shallower depth of field than physical aperture
• Depth-independent defocus blur size
Notas del editor
The technique was pioneered by radar engineer Edward Dowski and his thesis adviser Thomas Cathey at the University of Colorado in the United States in the 1990s. "Flexible Depth of Field Photography," H. Nagahara, S. Kuthirummal, C. Zhou, and S.K. Nayar, European Conference on Computer Vision (ECCV), Oct, 2008. Motion-Invariant Photography Anat Levin Peter Sand Taeg Sang Cho Fredo Durand William T. Freeman Computer Science and Artificial Intelligence Lab (CSAIL) Massachusetts Institute of Technology