An Augmented Reality “Pre-Cog.” vision system. This vision-augmentation system aims to assist the human ability to predict the short-term future by visualizing trajectory information of real-world objects via an optical see-through head-mounted display and a projector. Project: http://im-lab.net/LaplacianVision Video: https://youtu.be/NNLelFnU0nw Publications: - Yuta Itoh, Jason Orlosky, Kiyoshi Kiyokawa, and Gudrun Klinker. 2016. Laplacian Vision: Augmenting Motion Prediction via Optical See-Through Head-Mounted Displays. In Proceedings of the 7th Augmented Human International Conference 2016 (AH ’16). ACM, New York, NY, USA, , Article 16 , 8 pages. DOI=http://dx.doi.org/10.1145/2875194.2875227. Best Paper Award - Yuta Itoh, Yuichi Hiroi, Jiu Otsuka, Maki Sugimoto, Jason Orlosky, Kiyoshi Kiyokawa, and Gudrun Klinker. 2016. Laplacian vision: augmenting motion prediction via optical see-through head-mounted displays and projectors. In ACM SIGGRAPH 2016 Emerging Technologies (SIGGRAPH ’16). ACM, New York, NY, USA, , pages. DOI: http://dx.doi.org/10.1145/2929464.2949028

1. Laplacian Vision:
Yuta Itoh, Yuichi Hiroi, Jiu Otsuka, Maki Sugimoto,
Jason Orlosky, Kiyoshi Kiyokawa, Gudrun Klinker
Augmenting Motion Prediction via
Optical See-Through Head-Mounted Displays and Projectors

2. Photo: Steve-65
Naïve Physics (Hayes et al. 1978):
our untrained perception of basic physical phenomena

3. In daily life…
Photo: AEQWF

4. Naïve physics is, however, imperfect
http://www.failepicfail.com/

5. In Science Fiction, a precog sees his short-term future…
Next, Nicholas Cage, TM & © Paramount (2007)

6. …or brain implants enable you to calculate it
Jump 225 Trilogy, David Louis
Edelman, 2006-2010
Wizard’s brain, Reiichi Saegusa,
2001-present

7. Laplace's demon: a causal determinism
If the Demon knows the precise
location and momentum of
every atom in the universe,
Essai philosophique sur les probabilités, Pierre-Simon Laplace, 1814
their past and future values
for any given time are entailed;
they can be calculated from
the laws of classical mechanics.

8. Laplacian Vision: a vision augmentation system
to assist motion prediction.
Optical See-Through HMDShot directly through the system

9. Related work:“Aftermath”(AR + physics prediction)
Leigh & Maes CHI’15

10. Related work: projection mapping
Koike & Yamaguchi,
AH’15
Projector
Charette et al., ICCP’12
Custom DLP projector
Okumura et al., ICME’12
Saccade Mirror

11. Related work: human vision enhancement
Orlosky et al. TVCG’15
Field of view
Itoh & Klinker AH’15
Eyesight X-ray vision
Avey et al. VR’09

12. Our goal is to assist our naïve physics ability by
augmenting our vision with an Optical See-Through HMD

13. We built a proof-of-concept system, and track a flying ball
Tracking system
User
OST-HMD
Virtual
Image

14. We built a proof-of-concept system, and track a flying ball
Tracking system

15. Projection mapping add-on by a projector

16. Projection mapping add-on by a projector

17. Our system estimates the future path of a
flying ball in real time,

18. then visualizes an estimated path on the OST-HMD

19. Some views from the system when placed different positions

20. Some views from the system when placed different positions

21. Summary
 Assisting our naïve physics skill
 Optical See-Through HMDs
with a real-time AR rendering
 Simulating the physical behavior
of the user’s environment

22. x Physics
Human x OST-HMD
Laplacian Vision:
Yuta Itoh, Yuichi Hiroi, Jiu Otsuka, Maki Sugimoto,
Jason Orlosky, Kiyoshi Kiyokawa, Gudrun Klinker
Augmenting Motion Prediction via
Optical See-Through Head-Mounted Displays and Projectors

23. Appendix

24. Example 1

25. Example 2

26. Example 3

27. User study: a fall-point prediction game
With Laplacian vision Without (i.e. the naïve vision)
29 subjects * 5 launches * with/without

28. 80% 90% Impact
Video Length conditions
29 subjects * 5 launches * with/without * 3 lengths

29. 0° 90°45°
View Angle conditions
29 subjects * 5 launches * with/without * 3 lengths * 3 angles

30. Result: 3x accuracy improvement
0° 90°45°
Bird’s-eye view

31. Proposed
Baseline
45° 0° 90°
80%
90%
100%
View Angle
Video
Length
** ** **
** ** **
** **

32. IR image
(60Hz)
Tracking
(2x ARTTRACK2)
3D position of
the ball (60Hz)
Target
ball
System B
Prediction & Rendering
(Unity 3D)
Real world
System A
OST-HMD
(nVisor ST60)
~19 ms
latency
~80 ms
latency
(User-view camera)
Observer
Displayed image (60Hz)
Rendered image (60Hz)
Latency diagram

33. Forward prediction
100 ms forward prediction
No prediction
Real ball
carrying

34. HMD Display Calibration
World
Eye
(a user-view camera)

35. Single Point Active Alignment Method (SPAAM) [Tuceryan ’00] [Genc ’02]
Align a 3D point in the
scene to 2D screen points
A.R.T tracking system

36. Hardware