2. Outline
VR vs. AR : Is AR Hype?
Where are we now?
What’s Next?
Why Ubiquitous VR, DigiLog & CAMAR?
Summary and Q&A
3. Virtual Reality vs. Augmented Reality
Virtual Reality (out there):
Puts people in a computer-generated world
Augmented Reality (here):
Puts virtual content in a real physical world
Ubiquitous VR : VR over ubiComp Env (here & right now):
Forces the VR/MR/AR to live out here in the world with people
Johnny Mnemonic Minority Report
4. Augmented Reality vs. Virtual Reality
AR VR
Supplements reality, rather than Completely immerse a user inside
completely replacing it a synthetic environment
User maintains a sense of presence While immersed, the user cannot
in real world (Here and Right Now!) see the real world around him (Out
there!)
• Apply to all senses, not just sight: sound, haptics, etc.
• Extend and enhance user’s ability (performance/perception of the
world)
5. What’s AR?
What’s Augmented Reality?
It is really an old idea, e.g. “Terminator vision in 1990”
Milgram's Reality-Virtuality Continuum [94]
Azuma's definition on AR [97]
combines real and virtual
is interactive in real time
is registered in real 3D world
6. AR is Hype?
The web is dead. Long Live the Internet!
Web
Aug. 2010
7. AR is Hype?
Google Trend (VR vs. AR)
A: Virtual Reality Embraced by Businesses
B: Another use for your phone: 'augmented reality
C: Qualcomm Opens Austria Research Center to Focus on Augmented Reality
D: Qualcomm Launches Augmented Reality Application Developer Challenge
E: Review: mTrip iPhone app uses augmented reality
F: Toyota demos augmented-reality-enhanced car windows
8. AR is Hype?
Half century after its birth, the VR is in decline.
Two decades after its birth, the AR is in saturation.
AR over web for interoperability?
Smart AR Apps for just-in-time services?
AR authoring and sharing for sustainable eco-system?
So, will AR be dead?
@ Wired September 2010
9. Hype Cycle of AR
LBS-based AR I might be able to
make money
Mobile 2D AR Nope, no I am wondering
money here how I lived
without AR
3D AR • Mobile IT Infra
• CV HW/SW Tech
I hate AR • AI (context) Tech
Mobile 3D AR
Marker-based AR
CAMAR 2.0
What is CAMAR? U-VR?
10. Hype Cycle of AR 2011
Augmented Reality
• MIT’s annual review; “10 Emerging Tech.s 2007”
• Gartner: top 10 disruptive tech 2008-12
2010 • Juniper: mAR 1.4B downloads/y, revenue $1.5B/y
by 2015 (11M in 2010)
2009
2008
11. Outline
VR vs. AR : Is AR Hype?
Where are we now?
What’s Next?
Why Ubiquitous VR, DigiLog & CAMAR?
Summary and Q&A
12. Augmented Reality
What is involved in the AR process?
Get video from
camera
Estimate
Update the position and
status orientation of
the camera
Render the
Process the
augmented
interaction
scene
13. AR Trend
3D vs. Informative AR
Items 3D AR Informative AR
Main focus Realistic 3D augmentation Information augmentation
Input Camera Sensors (GPS, Compass, etc.)
CoI Object Location (POI)
Core Tech Object recognition/tracking POI detection
3D registration Overlay
Realistic 3D rendering Information filtering
Interaction 3D virtual objects vs. 2D text vs. user
physical space & user
Authoring Offline In-situ
Killer apps Games, training, education Navigation, adv, sns
Platform Standalone Networked
PC-based Sony, Total Immersion WearableAR with HMD
Phone-based Vuforia(Qualcomm), Layar, wikitude, sekaiCam,
Junaio/iLiving(metaio), Argon
16. Digilog Apps 2010
Enhance Experience, Engage, Educate & Entertain
Hongkil Dong Technologies in Chosun
Storytelling application Storytelling application
Integrated with virtools* Integrated with virtools*
17. Digilog Apps 2011
DigiLog Miniature
Hong-Kildong Asadal-Asaneo
Storytelling application Storytelling application
Desktop AR Desktop AR + mobile interaction
Integrated with virtools* Integrated with virtools*
18. Why Mobile AR?
Why phone-based AR ?
968M(40M) m-phones worldwide (Korea) in 09
64M(15M) s-phone worldwide(Korea) in 11
Low cost, ubiquity, robust, self-contained
From terminal to gateway:
With advances in tracking and increased
computing power, mobile AR systems are
developing
Leading graphics board vendors release new
chipset that will enable HW-accelerated 3D CG http://www.capv.com/home/Multiclient/MobileImaging.html
on mobile phones
AR Apps in iPhone AppStore
54 in July, 2010 and 605 total. (260K Apps in
Oct., 2010)
19. Mobile UI/UX
Camera for Physical Interaction
Connecting People, Places, Things, and Ideas
The environment as part of the interface
Camera phones as “bridging” devices
Integration with the user’s activities
Click real world !!!
20. Mobile AR : Informative
Informative AR Applications
Wikitude Layar TwitAround
Junaio Aurasma vuforia
21. Informative mobile AR
How does it work?
Illustrations courtesy of GeoVector (left)
and Layar & Junaio by Metaio (right)
22. Mobile AR SDK
QualComm SDK:
http://developer.qualcomm.com/dev/augmented-reality
https://ar.qualcomm.com/qdevnet/
23. Mobile AR SDK
KHARMA by GATECH
KML/HTML Augmented Reality Mobile Architecture
An Open Platform for Delivering Mobile Augmented Reality Experiences
https://research.cc.gatech.edu/polaris/
CAMAR 2.0 by GIST
Context-aware mobile AR
CAMAR + LBS + SNS
CONTEXT
SNS
LBS
CONTENT
Embedded Sensors
24. Outline
VR vs. AR : Is AR Hype?
Where are we now?
What’s Next?
Why Ubiquitous VR, DigiLog & CAMAR?
Summary and Q&A
25. What is still holding us back?
Processing power <10fps ~30fps
Registration accuracy 1cm 1deg
Screen size Limited FOV Optical-see-through
Interaction Limited to touching NUI with 5 senses
screen
GPS Limited speed and AGPS and wifi with
accuracy Vision-assisted
Network BW/latency 400ms pings are not so <50ms for cloud
responsive
Authoring Just emerging In-situ mash-up
Battery power Devour batteries Full day usage
Platform Dual core smart phone Quad core smart phone
Apps Navigation M-commerce
26. What to Expect in 2012?
Matthias Galica (CEO of ShareSquare)
Quick Response (QR) Codes : simple yet effective…
Augmented Reality (AR)
Near Field Communication (NFC)
The Field (Everybody Else)
The effects of AR
Increased transparency
Reduced privacy
Timely access to information
Juniper Research report on AR
predicted that annual revenues from mobile AR apps will reach
$732M by 2014,
$1.5B by 2015,
up from less than $1M in 2009.
27. What to Expect in 2012?
Where is Digital World headed?
Wearable, Cloud computing
Internet of things with sensor proliferation
Digital immersion with Rich UI/UX over ubiComp
SNS over networked urbanism
SNS needs AR!
More than AR needs SNS, SNS needs AR to improve the UI/UX!
Augmented Content is a King, then Context is a queen consort controlling the King!
So, ubiComp and AR are tightly coupled ‘Outernet’!
Mobile : smart phone for mobility of people
Social : layers of AR for cultural/collaborative experience
Cloud : bringing down all the resources of the internet to reality
eco-system: publishing sharable cloud content
28. How to make Mobile AR App’s useful?
Beyond the hype:
Moving from simple gimmicky experiments to real world applications
Current AR experience
shows generally low social maturity
Is stalling, waiting for breakthrough either technological or conceptual
Future Impractical AR
AR needs to Useful AR
•3D models placed in a webcam of replacing them with AR gimmicks
Enhance social interactions, instead •Connectivity is always ON
with optional for social interaction
Be little or no interactivity •The cloud is the computer
•Layered animation with little or no •Engaging, persistent experience for
feedback the user
•MAR that uses solely GPS, compass,
and accelerometer input •[LBS + SNS + MAR] drawing from
networked large DBs with
•MAR where geo-tagging doesn't customization features
serve an everyday purpose
30. Social AR?
Issues of Social AR
Physical self along with a digital profile
Unauthorized Augmented Advertising
Privacy: Augmented Behavioral Targeting
Safety: Physical danger
Spam
32. Outline
VR vs. AR : Is AR Hype?
Where are we now?
What’s Next?
Why Ubiquitous VR, DigiLog & CAMAR?
Summary and Q&A
33. Smart Holistic Space
Convergence
Virtual space
Between Real & Virtual
Seams btw two worlds How to
combine
Seamlessly?
Real space
Space for experience and expression
Personal space (3rd skin): an individual has a complete control of data/info flow, i.e.,
incoming, filtering, selecting, outgoing (using a user profile)
Social space: a group has some control, ownership and regulatory power, but as a
result of negotiations and transactions
General space: a temporary quality and free access, but regulated by a public social
contract
34. What’s VR, AR and UVR?
Dual space {R, R’}
RE RE’
V
RE
RE
R R’
VE’
RE RE’
VE’
35. What’s Ubiquitous VR?
Woo’s Definition [11] : U-VR is
3D Link btw dual (real & virtual) spaces with
additional info
CoI augmentation, not just sight: sound,
haptics, smell, taste, etc.
Bidirectional UI for H2H/H2S/S2H/S2S
communication in dual spaces
Virtual space
How to U-Content
Seamless Augmentation
Link btw dual spaces
Seamlessly? LINK
CoI
Real space
Social Networks
36. AR for DigiLog Experience
AR platform for DigiLog Experience will be
How to avoiding the fate of VR ?
"to look for DigiLog, the intersections • Multicore/GPGPU
between the digital world (3D, SNS) • Object recognition/tracking
and analog world (CoI)” • Light source estimation
Realistic 3D • Physics engine
•
augmentation AI engine
Context-
awareness
• Interaction
• CoI-aware
• Mash-up authoring
• Environmental context: User • SNS
• User context: knowledge, participation • Extension of human
experience, preference
perception
• Just-in-time visualization
37. Technical Challenges
CoI Localization:
Context of Interest (CoI): Space vs. Object
Accurate CoI Recognition and Tracking
3D Interaction
Ubiquitous Augmentation
LBS/SNS-based Authoring and Mash-up
Smart UI for Intuitive Visualization
AR-Infography + Organic UI
Networking and public DB management
U-VR ecosystem with SNS, LBS, CaS
Interoperable representation
HW wish list
Better camera/GPS/compass, CPU/GPU, I/O, battery
38. CAMAR 2.0: Context-aware mobile AR
Sharing
Direct
response
Mashup
Reflective
response Planned
response
39. CAMAR 2009-10
Mobile AR: WHERE to augment?
Concept Context-aware Annotation (H. Kim)
Plan Recognition (Y. Jang) Multi-page Recognition (J.Park) LBS + mobile AR (W. Lee)
[Paper] Y. Jang and W. Woo, “Stroke-based semi-automatic region of interest detection for in-situ painting recognition", 14th International Conference on Human-Computer Interaction (HCII 2011), Jul. 9-14,
Orlando, USA, accepted.
[Patent] W. Woo, Y. Jang, “현장에서 그림 인식을 위해 선긋기 상호작용을 통한 반자동식 관심영역 검출 알고리즘 ,” 2010. (출원 중)
40. Context @ U-VR Lab 2010-11
Adaptive Content Recommendation
Recommend user-preferred content
Retrieve content efficiently using hierarchal
context model
J. Han, H. Schmidtke, X. Xie, and W. Woo, “Adaptive Content Recommendation using Hierarchical Context Model with Granularity for Mobile Consumer,” in Pers. Ubiqu. Comp
ut., pp.000-000, 2012. (Submitted)
41. Interaction @ U-VR Lab 2011
ARWand: Phone-based 3D Object Manipulation in AR
Exploits a 2D touch screen, a 3DOF accelerometer, and compass sensors information
to manipulate 3D objects in 3D space
Design transfer functions to map the control space of mobile phones to an AR display
space
Taejin Ha, Woontack Woo, "ARWand: Phone-based 3D Object Manipulation in Augmented Reality Environment," ISUVR, pp. 44-47, 2011.
42. Interaction @ U-VR Lab 2011
ARWand: Phone-based 3D Object Manipulation in AR
Experiment and application
◦ Low control-to-display gain: a sophisticated translation could be possible but this requires a significant amount of
clutching
◦ High gain could reduce the frequent clutching, but accurate manipulation could be difficult
◦ Therefore, we need to consider an optimal control function that satisfies both fast and accurate manipulation
Taejin Ha, Woontack Woo, "ARWand: Phone-based 3D Object Manipulation in Augmented Reality Environment," ISUVR, pp. 44-47, 2011.
43. CAMAR @ U-VR Lab 2010
In Situ Video Tagging on Mobile Phones
In situ Planar Target Learning on Mobile Phones
Sensor-based Automatic Fronto-parallel View Generation
Fast Vanishing Point Computation
Input Image
Horizontal Vanishing Points
target ? Estimation
Fronto-parallel View
Generation
Target Learning
on the mobile GPU
Real-time Detection
W. Lee, Y. Park, V. Lepetit, W. Woo, "In-Situ Video Tagging on Mobile Phones," Circuit Systems and Video Technology, IEEE Trans. on, Vol. 21, No. 10, pp. 1487-1496, 2011.
W. Lee, Y. Park, V. Lepetit, W. Woo, "Point-and-Shoot for Ubiquitous Tagging on Mobile Phones," ISMAR10, pp. 57-64, 2010.
44. CAMAR @ U-VR Lab 2010
In Situ Video Tagging on Mobile Phones
In situ Augmentation of Real World Objects Vertical Target Learning & Detection
데모비디오
- In situ augmentation of real world objects - Learning a vertical target from an arbitrary
without pre-trained database viewpoint
- Fast target learning in a few seconds - Vanishing point-based fronto-parallel view
- Real-time detection from novel viewpoints generation
- Real-time detection from unseen viewpoints
Available at : http://youtu.be/vaaFhvfwet8
W. Lee, Y. Park, V. Lepetit, W. Woo, "In-Situ Video Tagging on Mobile Phones," Circuit Systems and Video Technology, IEEE Trans. on, Vol. 21, No. 10, pp. 1487-1496, 2011.
W. Lee, Y. Park, V. Lepetit, W. Woo, "Point-and-Shoot for Ubiquitous Tagging on Mobile Phones," ISMAR10, pp. 57-64, 2010.
45. CAMAR @ U-VR Lab 2011
Interactive Annotation on Mobile Phones for Real and Virtual
Space Registration
Allows to quickly capture the dimensions of a room
Operates at interactive frame-rates on mobile device
and provides simple touch-interaction
Serves as anchors for linking virtual information to
the real space represented by the room
H. Kim, G. Reitmayr and W.Woo, “Interactive Annotation on Mobile Phones for Real and Virtual Space Registration,” in Proc. ISMAR 2011, pp.265-266, Oct. 2011.
46. CAMAR @ U-VR Lab 2011
Interactive Annotation on Mobile Phones for Real and Virtual
Space Registration
Demo #1 데모비디오 Demo #2
In office room and seminar room, In ART gallery,
- Capture the dimensions of a room, - Load an AR zone-based room model
approximated as a room - Annotate a virtual content on rectangular areas
- Annotate a virtual content on rectangular on the room’s surface
areas on the room’s surface
Youtube share link http://www.youtube.com/watch?v=I00I-phmPbI
47. CAMAR @ U-VR Lab 2011
In-situ AR Mashup for AR Content Authoring
Easily create AR contents from Web contents
Context-based content recommendation
User-similarity, item similarity, social relationship
Configure AR content sharing setting
To Whom, When, in What conditions
H.Yoon and W.Woo, “CAMAR Mashup: Empowering End-user Participation in U-VR Environment,” in Proc. ISUVR 2009, pp.33-36, July. 2009. (Best Paper Award)
H.Yoon and W.Woo, “Concept and Applications of In-situ AR Mashup Content,” in Proc. SCI 2011, pp. 25-30, Sept. 2011.
48. CAMAR @ U-VR Lab 2011
In-situ AR Mashup for AR Content Authoring
In-situ Content Mashup
• Extract query keywords based on context of object
• Content recommendation based on personal context and social context
• Access related Flickr, Twitter, Picasa contents in-situ
H.Yoon and W.Woo, “CAMAR Mashup: Empowering End-user Participation in U-VR Environment,” in Proc. ISUVR 2009, pp.33-36, July. 2009. (Best Paper Award)
H.Yoon and W.Woo, “Concept and Applications of In-situ AR Mashup Content,” in Proc. SCI 2011, pp. 25-30, Sept. 2011.
49. Some Issues: Standard
Interoperability (Standard)
W3C : HTML5 (ETRI): http://www.w3.org/2010/06/w3car/report.html
web3D : X3D (Fraunhofer)
OGC : ARML (GATECH)
ISO/IEC JTC1: SC24 WG9 [X3D(KU), XML(GIST)] vs. SC29 [X3D(ETRI)]
Real Content, Virtual Content, ***Link***
POI: Metadata Metadata Format
MR
id ID
2
id ID
who author
who owner, author
when creation time
when creation time
where location Real where real-virtual relation
what ID of real and virtual
what tag, description Contents contents
how movable, access how access
AR/MR Link Lookup Table Link Link Link Link
1
Virtual Virtual Virtual Virtual
Contents Content Contents Contents
#1 #2 #3 #4
id ID
AR Content: Metadata who owner, author
3
when creation time
where location
what tag, type
how movable, access
50. Standard by GIST
Server/DB Mobile Device Real World
Feature Points or Image Acquisition (Grab)
Content ID
-FP Recognition AR
-AR Content Management Viewer Object of Interest (OOI)
Context of Interest
WHER WHAT
LID WHO WHEN HOW
E
V-
R-ID
Image
V-ID R-ID, R-Pose (COI)
GUID1 … … VID1 … Point of Interest (POI)
Pose ID1
GUID2 …
V- Image
GUID3 … … Pose ID1
VID4 …
Link DB
TTAK.KO-10.0434 TTAK.KO-10.0428
Data Representation for QR Code Data Representation
AR/MR (=Link) for (Mobile) AR
51. What’s Next?
Next?
People don’t necessarily want to walk around holding phone in front of their faces
CAMAR 2.0 vs. Wearable CAMAR 2.0
<Open + Participate + Share>
LBS + In-situ Mashup + SNS + CAMAR for sustainable AR eco-system
52. Outline
VR vs. AR : Is AR Hype?
Where are we now?
What’s Next?
Why Ubiquitous VR, DigiLog & CAMAR?
Summary and Q&A
53. Q&A
“The future is already here. It is just not uniformly distributed”
by William Gibson (SF writer)
More Information
Woontack Woo, Ph.D.
Twitter: @wwoo_ct
FB: @wtwoo
Mail: wwoo@gist.ac.kr
Web: http://uvr.gist.ac.kr
1st UVR Workshop @ KAIST GSCT, Mar. 16-17, 2012
7th ISUVR 2012 @ KAIST GSCT, Aug. 22 - 25, 2012