Mark Billinghurst's talk on Building Usable AR Interfaces at the ARE 2012 conference, May 10th 2012

1. Building Usable AR Interfaces
Mark Billinghurst
HIT Lab NZ
University of Canterbury

2. The Vision of AR

3. To Make the Vision Real..
 Hardware/software requirements
 Contact lens displays
 Free space hand/body tracking
 Speech/gesture recognition
 Etc..
 Most importantly
 Usability

4. Usability Issues
 Perception
 How to make images appear part of real world?
 Interaction
 How to pick content in mid air?
 Social
 Do I look stupid?
 Cultural
 Wave up or down?
 Cognitive
 How do I remember where everything is?

5. “The product is no longer
the basis of value. The
experience is.”
Venkat Ramaswamy
The Future of Competition.

6. Gilmore + Pine: Experience Economy
experiences Emotion
services
Value
products/tools
Function
components

7. Build Experiences NOT Applications!

8. Building Compelling AR Experiences
?? experiences
Usability
applications Interaction
tools Authoring
components Tracking, Display
Sony CSL © 2004

9. The Interaction Design Process

10. Understand Your Users
 Workshops or focus groups
 Group interviews/activities
 Observations
 Spending time with users in day to day tasks
 Questionnaires
 Looking for specific information
 Interviews
 Good for exploring issues, using props
 Documentation
 Procedures and rules written down in manuals

11. Consider the Whole User

12. AR as Perception Problem
 Goal of AR to fool human senses – create
illusion that real and virtual are merged
 Depth
 Size
 Occlusion
 Shadows
 Relative motion
 Etc..

14. TAT Augmented ID

17. AR Rapid Development
Prototyping and User Testing
 Low Fidelity Prototyping
 Sketches
 Paper Prototyping
 Post-It Prototyping
 PowerPoint Prototyping
 High Fidelity Prototyping
 Wikitude, Junaio, Layar, BuildAR etc

18. POST IT PROTOTYPING
Camera
View
with
3D
First
Dra)
Second
Dra)
Third
Dra)
•
Selec8on
highlighted
in
•
Home
bu=on
added
for
blue
easy
naviga8on
to
main
menu

19. POWERPOINT PROTOTYPING
Benefits
•
Used
for
User
Tes8ng
•
Interac8ve
•
Func8onali8es
work
when
following
the
story
of
Scenario
1
•
Quick
•
Easy
arrangement
of
slides
User
Tes8ng
•
Par8cipants
found
•
15
minute
sessions
screen
captured
•
‘ Talk
Allowed’
technique
used
•
Notes
taken
•
Post-­‐Interview

20. BuildAR
 http://www.buildar.org/
 Stand alone application
 Visual interface for AR model viewing application
 Enables non-programmers to build AR scenes

21.  Interface Components
 Physical components
 Display elements
- Visual/audio
 Interaction metaphor
Physical Display
Elements Elements
Interaction
Metaphor
Input Output

22. AR Design Space
Reality Virtual Reality
Augmented Reality
Physical Design Virtual Design

23. AR Lens Design Principles
 Physical Components
 Lens handle
- Virtual lens attached to real object
 Display Elements
 Lens view
- Reveal layers in dataset
 Interaction Metaphor
 Physically holding lens

24. AR Chemistry (Fjeld 2002)
 Tangible AR chemistry education

25. Case Study: LevelHead
 Block based game

26. Case Study: LevelHead
 Physical Components
 Real blocks
 Display Elements
 Virtual person and rooms
 Interaction Metaphor
 Blocks are rooms

28. Goal: An AR application to test molecular
structure in chemistry
 Physical Components
 Real book, rotation cube, scoop, tracking markers
 Display Elements
 AR atoms and molecules
 Interaction Metaphor
 Build your own molecule

29. AR Chemistry Input Devices

31. Natural Hand Interaction
 Using bare hands to interact with AR content
 MS Kinect depth sensing
 Real time hand tracking
 Physics based simulation model

32. Evaluation
 Need for more evaluation
 2008 -10% AR papers in IEEE, ACM had any evaluation
 Informal
 Pilot, ‘quick and dirty’
 Formal
 Lab studies, field studies, heuristic

33. 2D vs. AR Navigation?
VS

34. HIT Lab NZ Test Platform – AR View

35. HIT Lab NZ Platform – Map View

36. Distance and Time
No significant differences

37. Paths Travelled
 Red – AR
 Blue – AR + Map
 Yellow - Map

38. Navigation Behaviour
 Depends on interface
 Map doesn’t show
short cuts

39. Survey Responses

40. User Comments
 AR
 “you don't know exactly where you are all of the time.”
 “using AR I found it difficult to see where I was going”
 Map
 “you were able to get a sense of where you were”
 “you are actually able to see the physical objects around you”
 AR+MAP
 “I used the map at the beginning to understand where the
buildings were and the AR between each point”
 “You can choose a direction with AR and find the shortest way
using the map.”

41. Building Usable AR Interfaces
 Understand user needs
 Consider whole user needs
- Physical, emotional, cognitive, social, cultural
 Perceptual issues
 Design for those needs
 Rapid prototyping
 Virtual, physical elements, interaction metaphor
 Test your design
 Formal, informal testing

42. More Information
• Mark Billinghurst
– mark.billinghurst@hitlabnz.org
• Website
– www.hitlabnz.org
– www.buildar.org