The document discusses potential alternatives to the current mobile interaction paradigm which is based on touchscreens. It proposes using gestures for interacting with a mobile search app, specifically for a library catalog search. It outlines some example gestures that could be used like tilting or rotating the phone. The document then demonstrates a prototype mobile search app for a library that incorporates gestures. It acknowledges challenges with developing gesture controls including a lack of standards and difficulties implementing more complex gestures. The discussion focuses on moving beyond just touchscreens to explore new interaction methods using a phone's sensors and motion.

1. Kill the Search
Button II
The Handheld
Devices Are
Coming

2. Gestures for a Mobile Search App
Michael Poltorak Nielsen

3. Today's agenda
● Current Mobile Interaction
Paradigm
● An Alternative
● Smartphone Gestures
● Demo: A Library Search App
with Touch and Gestures

5. But, the Current Handheld Interaction Paradigm
is based on:

6. Glass

9. The future
is sliding,
too
microsoft.com/
office/vision

14. Pictures Under Glass
Transitional
paradigm dictated by
technology, not
human capabilities
Bret Victor
worrydream.com/
ABriefRantOnTheFutureOfInteractionDesign

15. Alternative Paradigm
Direct Manipulation
Gesture Driven
Palpable
Tactile

19. Harrison, C. and Hudson, S. E. 2009. Providing Dynamically Changeable Physical Buttons on a Visual
Display

20. Not bad, huh?

21. Near Future?
+

22. Have a Moment

23. Our Mobile Projects

24. Mobile Search App(s)
HTML5 + iPhone app, Android?
● search library data (10M + 800M)
● request stuff, favourites
● view own items
● renew
In beta, to be published
really soon

25. Barcode to Map Location App
With Silkeborg Public Library

26. Now: Gestures
● Augment search app with
gestures
● Can gestures be combined with
multi-touch interactions?
● Goals:
○ Produce alpha app
○ Test usability

27. Smart Phone Interactions and Inputs
Keyboard typing
Microphone speech
Screen touch, visuals
Camera pattern, movement
Accelerometer acceleration
Gyroscope rotation
Compass direction
GPS movement, position

28. Focus on
Keyboard typing
Microphone speech
Screen touch, visuals
Camera pattern, movement
Accelerometer acceleration
Gyroscope rotation
Compass direction
GPS movement, position

29. Gestures
Simple Advanced
Tilt, flip, turn, rotate, Circle, cross, up-
shake, throw down, etc
Accelerometer data Movement hard to
measure
Can't reliably calculate movement
from acceleration

30. What Can Browsers do
Accelerometer-wise?

31. So: Going Native
iPhone app with gestures
HTML5 web app without
gestures
Android?

32. Search App Interactions
Do search query, toggle search box
Switch views compact, normal, expandend
Next/previous page switch search results
Show full record full info about item
Request get the material
Favourites save material

39. Demo Time

40. What's next?
From Nokia

41. Challenges
No standard mobile gestures
Gestures may be individual
Gestures may not be appropiate at all
Sophisticated gestures hard to code
Objective-C Argh!

42. Thank you!
Michael Poltorak Nielsen
mn@statsbiblioteket.dk
twitter:@michaelpoltorak