Keynote at AVI 2022: Advanced Visual Interfaces, Rome, Italy, 6-10 June 2022 https://alandix.com/academic/talks/ AVI2022-keynote/ As cultures and individuals, we look back to look forward, explicitly using lessons of the past to guide our future decisions. In this 30th anniversary of the AVI conference we can also look back in order to make the next 30 years even better. In 1996 I gave a keynote at the third AVI, both describing my own work in formal modelling of user interactions, and also pondering variations of ‘advanced visual interfaces’, imagining advanced aural and advanced nasal interfaces. This was part playful but also serious, uncovering the way our different senses give us different cuts through space and time. In particular, smell is deeply associated with memory both personal and spatial – that is history. In the meantime, scent has been used as a metaphor for information seeking, and now long-promised smell-based interfaces are beginning to emerge; we are about to enter an exciting world of multi-sensory experiences. However, it is on the metaphoric sense of advanced visual and nasal interaction that I want to focus now. The importance of tracing what is past and planning what is to come. This is true for our discipline as a whole, but also in moment-to-moment digital interactions. On the first of April this year, Nielsen Norman Group posted an article entitled “Support Recall Instead of Recognition in UI Design”. It was meant as an April Fool’s Day joke, but in fact the move towards gesture-based touch interactions means this is precisely how many feel today. This has exacerbated the long-term weaknesses of visual interaction heuristics and guidance when it comes to looking back, making it hard to ask, “why did that happen?”, or “how did I manage that?”, especially for older users or those who are less confident with digital technology. Finally, as we move forwards to tackle these and new issues, we need to constantly question what constitutes ‘advanced’: a fast-moving highway for a few or a wider frontier for everyone. The latter often poses the hardest design challenges but is most critical in a world where being digital is central to being a citizen.

1. Follow your nose:
history frames the future
Alan Dix
Director of the Computational Foundry
Swansea, Wales
http://alandix.com/academic/papers/AVI2022-keynote/
@alanjohndix @CompFoundry

2. 30 Years of AVI

3. AVI’96
my first keynote
advanced visual interfaces
visual?

4. visual perception

6. nasal perception

8. audio perception (echolocation)

9. audio everywhere

10. … and smell technology
https://thehustle.co/digiscents-ismell-fail/
https://multi-sensory.info/index.php/component/spsimpleportfolio/item/11-owidgets

11. time and space
visual
one time – everywhere
nasal
one place – all time
… history

12. https://commons.wikimedia.org/wiki/File:IBM_PC_DOS_1.0_screenshot.jpg
https://en.wikipedia.org/wiki/Mac_OS_8#/media/File:Mac_OS_8.1_emulated_inside_of_SheepShaver.png
in 1996 … CLI (nasal) vs. GUI (visual)

13. undo and web history

14. https://commons.wikimedia.org/wiki/File:IBM_PC_DOS_1.0_screenshot.jpg
https://en.wikipedia.org/wiki/Mac_OS_8#/media/File:Mac_OS_8.1_emulated_inside_of_SheepShaver.png
that was 1996 …

15. it all seems so quaint now …
https://en.wikipedia.org/wiki/IOS_13

16. Growing choice
1970s & 1980s
– computers for work
– employer chooses
1990s & 2000s
– home computing and the web
– user chooses

17. Now …
digital access assumed
for commerce, education, government
digital is essential
to be a full citizen

18. mobile / touch for older users
fingers …
– precision (fat fingers),
wet, dry
icons
– what do they mean

19. Helping Hand
guiding mobile app use
without getting too close

20. most apps are
designed for a single
person to use.
helping can be
intrusive when
together …
… and even harder
when remote

21. even worse if
you want to
show someone
what to press.
get out
of my
space!
NEXT

22. imagine
user’s screen
is shared
with helper
perhaps
side-by-side,
across a table,
or in different
continents.
user
helper
NEXT

23. show me!
helper points
at what to do
on their screen
the same point
highlights on
user’s screen
for them to
press
user
helper
NEXT

24. Helping Hand
• user stays in control
• enables implicit learning
• no invasion of personal space
• helping at a distance

25. why do we need this?

26. four golden rules
knowing …
• where you are
• what you can do
• where you are going
– or what will happen
• where you’ve been
– or what you’ve done

27. four golden rules
knowing …
• where you are
• what you can do
• where you are going
– or what will happen
• where you’ve been
– or what you’ve done

28. https://www.nngroup.com/articles/recall-beats-recognition/

29. no joke for many …
https://pixnio.com/media/apple-computer-blonde-hair-famous-mobile-phone-woman

30. what gesture?
AVI 2010 … in Rome
Daniel Wigdor. Architecting next-generation user
interfaces.
Microsoft Surface (the table!)
ways to help teach gestures during use
… a bit like WordStar 

31. Wordstar

32. four golden rules
knowing …
• where you are
• what you can do
• where you are going
– or what will happen
• where you’ve been
– or what you’ve done

33. why did that happen?
how did I manage that?

34. Time ….
1987 Myth of the infinitely fast machine
… mostly keyboard

37. Time ….
1987 Myth of the infinitely fast machine
… mostly keyboard
tap vs hold
mute wars

38. mute wars
unmute
×

39. unmute
× unmute
×
mute
mute
unmute
×
YOU CHAIR
unmute
× chair
clicks first
unmute
you click
unmute
×
mute
unmute
×
mute
mute

40. Lockout button for short period during change
Lock original button – transmit state, not ‘toggle’
mute
mute
fixing it?
unmute
×
unmute
×
unmute
×
Chair: unmute
×
Chair: unmute
unmute
×
unmute

41. fixing it ????
… maybe more
nasal interfaces

42. nasal interfaces for touch-based interaction?
history lists
also useful for undo
c.f. Apple Events and workflows ..
replay
how?
open architectures
event/data layers within apps

43. in summary … looking back …
things that are new
lessons we’ve forgotten
things that need to be re-interpretted