Ramesh Raskar discusses special effects in the real world and spatially augmented reality. Spatial augmentation uses projectors, sensors and RFID to fuse virtual graphics with the real world in real-time. This allows for changing appearances and illumination. Overcoming challenges like complex reflectance, other senses, and photorealism could lead to believable virtual effects impacting daily life. Spatially augmented reality provides solutions but requires hardware advances.

1. Ramesh Raskar Mitsubishi Electric Research Labs (MERL) Cambridge, MA Special Effects in the Real World

3. Changing Appearance Virtual light source

4. Changing Virtual Illumination

7. Changing Appearance Projector Projector Virtual light source

8. Changing Virtual Illumination Raskar, Welch, Low, Bandyopadhyay, “ Shader Lamps ” (2000)

9. Virtual Motion

10. Virtual Motion Raskar, Ziegler, Willwacher, “Cartoon Dioramas in Motion,” (NPAR 2002)

11. Dynamic Augmentation Projecting on Tracked Objects

12. ShaderLamps Virtual Reflectance Virtual Illumination Interaction Virtual Motion www.ShaderLamps.com

13. Projector-based Augmentation www.ShaderLamps.com Virtual Reflectance Virtual Illumination Interaction Virtual Motion

14. Poor Man’s Palace

16. Augmented Reality (AR) With See-through Head-mounted Display © Andrei State

17. Head-attached Body-attached spatial Spatial Augmented Reality Classification of Augmented Reality hand-held display spatial see-through display head- mounted display retinal display projector projector projector

19. Desired Virtual Model © Andrei State Projected Guidance for Placement

21. Augmented Reality Head-mounted Display Projectors © Andrei State

24. Spatially Augmented Reality (SAR) HMD-VR Spatially Immersive-VR AR using HMD Spatially Augmented Reality e.g. CAVE Video or Optical see-through SAR, Shaderlamps

26. Spatially Augmented Reality Raskar, vanBaar, Beardsley, Willwacher, Rao, Forlines ‘iLamps: Geometrically Aware and Self-Configurable Projectors’, SIGGRAPH 2003

32. Warehousing Routing Library Baggage handling Currency Livestock tracking RFID

33. Conventional Passive RFID

35. Handheld Projector iLamps 2002 RFIG Lamps 2003-04 Pocket Projector 2004-05

36. Mitsubishi Pocket Projector

38. Prototype Tag RF tag + photosensor

39. Conventional RF tag Photo-sensing RF tag

41. Find tag location using handheld Projector Photosensing Wireless Tags Many geometric ops R F I R F I D Interactive stabilized projection (Radio Frequency Id & Geometry) G Siggraph 2004

42. AR with Photosensing RFID and Handheld Projector

50. RFID (Radio Frequency Identification) RFI G (Radio Frequency Id and Geometry )

51. Projectors and Surfaces Curved Planar Non-planar Single Projector Multiple Projectors Pocket-Proj Objects 2000 2000 2002 2001 1999 2002 2003 1999 1998 Projector j User : T ?

52. Pieces of the Puzzle Actuated Surfaces Actuated Workbench [Pangaro, Maynes-Aminzade, Ishii UIST 2002]

57. Electric Cutaneous Display Skin Nerve Electrode Dr. H. Kajimoto, U Tokyo Masahiko Inami (UEC)

58. Smart Touch Masahiko Inami (UEC)

59. Scent Display Yasuyuki Yanagida, ATR Spatio-Temporal Controlled Delivery of Scent

60. Air Cannon Generating a doughnut-shaped vortex ring Aperture Push

61. Air Cannon

63. Beyond White Objects Grossberg, Nayar et al

65. Grossberg, Nayar et al

66. The Virtual Showcase Oliver Bimber Tracking and Mirrors

67. Displays Contenders Organic LED Light Emitting Polymers E-Ink

68. Holodust Perlin et al

69. Fake World