Paper presented at the 2018 International Conference on Digital Health. Available at https://doi.org/10.1145/3194658.3194676

1. Inferring Visual Behaviour from User Interaction
Data on a Medical Dashboard!
Ainhoa Yera, Javier Muguerza, Olatz Arberlaitz, Iñigo Perona
Faculty of Informatics – University of the Basque Country UPV/EHU
Richard Keers, Darren Ashcroft
Division of Pharmacy and Optometry – University of Manchester
Caroline Jay, Markel Vigo
School of Computer Science – University of Manchester
Richard Williams, Niels Peek
Division of Informatics, Imaging and Data Sciences – University of Manchester
Paper available at https://doi.org/10.1145/3194658.3194676

2. Motivation!
• Health data + software
• Proactive management of population health care
• SMASH a primary care intervention
– Medical dashboard
– Patient safety
– PINCER indicators1
• Conditions vs medications: CKD and NSAIDs
• Demographics vs medications: Woman, smoker, +35, CHC
– Pharmacists and GPs
– Deployed in Salford (UK)
1: Avery et al. (2012) A pharmacist-lead information technology intervention for
medication errors. The Lancet 379
1

3. SMASH screens!
5 - Trends
3 - Visualisation
1 - Landing page
4 - Patients
6 - Information
2 - Indicators
2

4. Problem!
• Information overload problem
• Addressing high information density in medical
decision making tools?
• “The right amount of information”
• Adaptive user interfaces
– What are these information needs?
– What’s taxing decision making of clinicians?
• Visual behaviour is proxy of interest and
information overload
“I just need the right amount of information”
Louise, GP at Greater Manchester
3

5. Visual behaviour as a proxy of interest and
information overload!
4

6. Problem!
• Adaptive user interfaces
– What are these information needs?
– What’s taxing decision making of clinicians?
• Visual behaviour is proxy of interest and
information overload
• Eye-trackers not expected beyond the lab setting
• RQ: can we link user interaction data to visual
behavior?
“I just need the right amount of information”
Louise, GP at Greater Manchester
5

7. Collected data!
• Laboratory (N=6)
– Interaction logs and eye-tracking data
– Typical tasks
• Remote (N=35)
– Interaction logs
– 10 months data (Jan-Oct 2016)
6

8. Computed metrics!
• User interaction metrics
– Exploration: mouse hovers between mouse clicks
– Pace: elapsed time between two consecutive mouse
clicks
• Gaze metrics
– Fixation duration: values between 50-400ms
• Vectors of aggregations
– V1: Exploration and pace on SMASH (length: 2)
– V2: Exploration and pace per screen (length: 7x2)
– G: Median fixation duration per AOI (length: 9)
7

9. Analysis: laboratory participants!
• Finding individuals with similar interactive behaviours
– Clustering analysis on log data (V1 and V2)
• k=3 optimal when applying silhouette analysis on Cluster Validity Indexes, k: 3..10
• Euclidean distance
• Same results for V1 and V2
• Hierarchical clustering including the centroids
• Neighbour-joining method
k-means, k=3, Euclidean
Log data: clustering on V
Log data: clustering on V
P3
P1
P4
P5
P6
P2 P1
P2
P3
P4P5
P6
P6
P2
C3
P4
P5
C1
P3
P1
C2
0.00.20.40.60.81.01.21.4
Cluster Dendrogram
DM_dmedEu2
Height
8

10. Analysis: laboratory participants!
• Finding individuals with similar interactive behaviours
– Correlation analysis on fixation durations (G)
– Same pairs as those emerging from interactive behaviour
Log data: clustering on V
Log data: clustering on V
P3
P1
P4
P5
P6
P2
Log data: clustering on V
Gaze data: correlation on G
P3
P1
P4
P5
P6
P2
9

11. Analysis: lab + remote participants!
• Finding individuals with similar interactive behaviours
– Lab (N=6) and remote (N=35) altogether
– Clustering analysis on log data, Euclidean distance
– k=3 optimal when applying silhouette analysis on Cluster
Validity Indexes, k: 3..10
All= lab + remote (N=41)
Log data: clustering on V
P1
P2
P3
P4P5
P6
P6
10

12. Laboratory (N=6)
All= lab + remote (N=41)
Gaze data
Log data
Main outcome/hypothesis: those individuals belonging to a
cluster in the remote group have similar visual behaviours to that
of laboratory users
Log data
Hypothesis: Gaze data
11

13. Outcomes!
• In terms of interactive behaviours clusters indicate
– P2 and P6: slow pace and high exploration
– P1 and P3: high pace and high exploration
– P4 and P5: high pace and low exploration
• In terms of visual behaviours (ie cognitive load)
– P2 and P6: less load when looking at indicators
– P1, P3, P4, P5: high cognitive load on indicators and data
table
12

14. Conclusions!
• There is a relationship between interactive behaviour
and visual behaviour
• We could infer visual behaviour by monitoring
interaction data
• Results not completely conclusive but promising
• We have a higher certainty
13

15. Questions?
Inferring Visual Behaviour from User Interaction
Data on a Medical Dashboard!
15
markel.vigo@manchester.ac.uk
@markelvigo