Talk at UCL Interaction Centre, London, 10th October 2018. http://alandix.com/academic/talks/UCL-more-than-a-moment-2018/ In understanding and designing effective and engaging interaction, we often focus on the moments of interaction the periods of minutes or hours while keystrokes, mouse clicks or finger movements across the screen elicit changing patterns of pixels and bits on computer memory. Yet these moments of actual interaction are part of a larger matrix of days, weeks and years, where periods of direct interaction string together to create larger patterns. This is the territory where user experience design meets service design, and HCI research meets IS. It is familiar to those working in CSCW where asynchronous interactions and workflows naturally take you beyond the system itself into the apparent interstices, that are, in fact, often the activities that interaction is about. The failures at this timescale are often errors of omission rather than commission, the things tardy, forgotten and undone. This is an area beyond the twenty minute user test; where motivation, opportunity and prompts to action, are more important than consistency, feedback and direct usability; where novelty may lie in the assembly of off-the-shelf applications; and success lies in life beyond the screen. In my personal work with various colleagues I have myself encountered and studied these long-term interactions over many years including trigger analysis for understanding cross-organisational processes, and extended episodic experience. For this talk I'll illustrate with more recent examples including using spreadsheets as interaction elements with musicologists, island community communication and an onion skin model of social and technology experience developed as part of the analysis of my 1000 mile round Wales walk.

1. more than a moment
Alan Dix
Computational Foundry
Swansea
http://alandix.com/academic/talks/UCL-more-than-a-moment-2018/

2. Tiree
Tiree Tech Wave
25-29 Oct 2018
Computational Foundry
Swansea University

3. the foundry
building
mission
community

4. computational foundry
opportunities
ECR programme
• emerging research leaders in the UK
• now in third year
escalator funds
• Swansea academic (not nec. computing)
• non-Swansea academic
• non-academic partner (industry, community, gov.)

5. today I am not talking about …
• algorithmic bias and AI explainability
• physicality and product design
• the long tail of small data
• deep digitality
• REF
• NOW
• digital light
• digital humanities
• creativity and Bad Ideas
• virtual crackers and slow time
• modeling dreams, regret and the emergence of self

6. I am talking about …
long-term interaction
hours, weeks and years
long-term research
first paper 1987
teasing out threads
you are part of it

8. triggers and
placeholders
why it happens when it happens

9. triggers
process – what happens and order
get post from
pigeon hole
bring post
to desk
open post

10. triggers
process – what happens and order
triggers – when and why
first thing in the
morning
holding post at coffee time
get post from
pigeon hole
bring post
to desk
open post

11. placeholders
• knowing where you are in a process
– like a program counter
• coding:
– memory
– explicit (e.g. to do list)
– in artefacts

12. lessons …
long-term interaction may
cross organisational boundaries
in long-term interaction
the gaps / breaks matter

14. extended episodic
experience
connecting threads of experience
based on Haliyana Khalid PhD on photologs
also Facebook experience (with Corina Sas et al.)

15. lots of focus on experience
but usually about
a single ‘experience’

16. much of our experience is:
extended – happening over protracted periods
but
episodic – composed of (linked) discrete events

17. “…I bothered to link them; they should bother
to link me. It’s like Friendster, we put
testimony, and they have to put testimony. If
this person didn’t link me, it’s like she is being
choosy of friends…”

18. interlinked events
each event is Janus-like
looks back to past experiences (retrospective)
looks forward to future ones (prospective)
c.f. McCathy & Wright

19. intertwined threads
threads of communication with people
repeated visits to places
crossing media
meeting one another
intermixing contexts
N.B. body chemistry shifts slowly

20. lessons …
human feelings and expectations
build across multiple events
threads of experience interconnect

22. the spreadsheet as
user interface
a musicological interlude
not just buttons and menus

23. interaction – using widgets
micro-interaction
managed by
widget
as interface
your clever
stuff
more clever
stuff
dialogue
design
parameter
selection

24. interaction – web form
more interaction
managed by
web form
as interface
your clever
stuff
more clever
stuff
high-level
dialogue design
web-form
design

25. interaction – spreadsheet
most interaction
managed by
spreadsheet
as interface
your clever
stuff
more clever
stuff
download upload
workflow
design
spreadsheet
design

26. musicological datasets in InConcert
upload & re-upload
entity matching between datasets
grouping within dataset

27. architecture and workflow
automatic
matching
visualise
in context
download upload
appropriate
intelligence
spreadsheet
design

28. print and annotate

29. lessons …
low-level interaction may be
outside the technical system
interactions are not always digital
appropriate intelligence for
mixed human–computer activity

31. island life
designing for the margins
low-intention design for
sensor-based systems

33. tidying
up
prepare
to open
prepare
to open
open sign off
open sign on
café
open
café
open
tidying
up
café
empty
(i) arrive
at café
café
empty
(a) forget
sign
(iii) open
café doors
(ii)
switch
on sign
(b) remember
sign
(iv) switch
off sign
(v) close
café doors
(d) remember
sign
(c) forget
sign
(vi) go home

35. 0010010
1010011
0100101
1110011
0110101
0001110
1000111
0101010
1101011
0010101
0101111
0101010
1101010
1101011
0101101
0101010
0111011
primary
data layer
application
data layer
API / RSS
web scrape
entry form
sensors
json AJAX data
for each app
text feed for
tickertape
0010010
1010011
0100110
1110010
0010011
1010011
0100100
1110011

36. lessons …
to design triggers and sensors
understand long-term activity
hard to
prime user data (zero-point problem)
hard to
maintain engagement

38. a walk in Wales
journeys are also social
even when solitary

39. space and place – absent presence
inhabited paths – traces

41. while walking
active
camera, recorder
passive
GPS, ECG, EDA
when stopped
iPad, blog, flickr,
social media,
uploading data
before/after
web for planning
new contacts on
Twitter, email
reporting, analysis
while walking
other walkers
occasional
friends/family
meeting
when stopped
shop keepers,
B&B proprietors,
chance meetings
talks, visits
before/after
family and friends,
readers,
researchers,
future walkers
social
sphere
technical
sphere

42. ‘normal’ life ‘normal’ life
walking life
universal time
April May June July2013
more than 3 month gap

43. universal time
April May June July2013
‘normal’ life ‘normal’ life
walking life
more than 3 month gap
event in past time of
remembering
underestimate
time

44. universal time
April May June July2013
‘normal’ life ‘normal’ life
walking life
more than 3 month gap

45. ‘normal’ life ‘normal’ life
walking life
more than 3 month gap

46. ‘normal’ life ‘normal’ life
more than 3 month gap
walking life
walking life

47. ‘normal’ life ‘normal’ life
walking life
‘normal’ life

48. lessons …
individual actions sit within
larger matrix of activity
large–scale activity consists of
different kinds of low-level actions
our memories are like spaghetti

50. academic life
actionable notifications
from learning analytics
matching the timescale

51. learning analytics
macro-analytics
university strategy
MOOCs
micro-analytics
individual course,
student,
resource

52. analytics and action …
action
?
?
?
recognise issues
current course
future course
allow
Micawber management
analytics
visualisation
automatic
drivers capability
value
career
development
resources
time
course materials
communication

53. time frames for learning analytics
days and hours
email, during lectures and labs, student meetings, gaps
week
preparing for teaching, exercises
months/mid-semester
reporting points, staff meetings, cohort/student progress
end of semester/term/year
exams, exam boards, course revew,
start of semester/term/year
preparing for new courses or re-runs, rollover!
years
new courses, professional development, appraisal, promotion

54. lessons …
to design notifications we must
map long–term patterns (hard!)
effective and actionable notifications
match pace (appropriate time-scales)
Micawber management

56. extending
walkthrough
extending the cognitive walkthrough
for multiphase interaction

57. multi-phase interaction
phases of direct interaction
gaps … but things happen

58. expert evaluation
standard:
cognitive walkthrough
heuristics
overall activity

59. 3a. overall aim/activity
Description of the overall activity within which the individual
phases of interaction fit, including aims/goals/drivers.

60. 4a. user story
Map out the overall phases of interaction with the system,
including relevant non-system activity before, between and after.

61. for overall activity or phases
conflict
– Are there any conflicts between organizational
and individual goals?
– If so are there means in the systems or supporting
process to deal with this?
emotion and motivation (if relevant)
– Is the system engaging/enjoyable
– how is motivation to use the system maintained

62. expert evaluation
start
standard:
cognitive walkthrough
heuristics
between phases maybe finish
overall activity

63. at start
knowledge
– How does the user know the system exists?
motivation
– What makes them want to use it?
opportunity
– What enables actual use? (installation, access etc.)

64. between phases
motivation
– What makes them want to re-engage?
trigger
– What prompts the user to re-engage?
opportunity
– How do they do so? (e.g. find URL, locate app)

65. at end
existence
– Is there a natural end to the overall activity
termination
– If user action is needed is this clear?
recognition
– Does the user know they have got to the end?
satisfaction
– Has the overall activity met the broad goals?

67. in summary …

69. long-term interaction
gaps matter, triggers & motivation,
multi-scale activity, pace,
cross-organisational, cross-media,
physical–digital, interlinked threads

70. long-term research
I am still learning