BELIV'10 Keynote: Conceptual and Practical Challenges in InfoViz Evaluations
A Descriptive Model of Visual Scanning.
1. A Descriptive Model of Visual Scanning
Stéphane Conversy1,2 Christophe Hurter1,2,3 Stéphane Chatty1
1 2 3
CNRS
INPT
UPS
UT1
Institut de Recherche en Informatique de Toulouse
1
3. flexible:
hand-written
efficient organization, multiple forms of layout on the board
however:
computer system unaware of clearances
only considered as a legacy system
«be modern, get rid of it !»
« all-radar » mantra 3
4. paper strip board considered as a
visualization
same route (no space Y),
different flights (Y:different) beacons (X inside:Ordered)
interacting flights (alignement)
orientation
landing flights runway 1(X:same Y:Ordered) landing flights runway 2 (X:same Y:O) 4
5. research question
my problem: how to explain that strips are better than
radar image for certain tasks ?
research question: how easy is it to accomplish a given
task with a given visualization ?
5
7. «how to integrate a flight?»
3.1
1 2.1 M
TBO
AGN
270
CRX92
C 28 11
21
M
NARAK
M
11 2.2
2.3
AF121ZL 100 FISTO LMG
142.4 26 2.4.2
LFBO 11 11
AF497PY 120 TAN AUCHE AGN PERIG
2.4.3.2
12 15 18 2.4.3.1
28
2.5
LFBT 11 11 11 11
SWR686 290 AGN TBO SOVAR
19
2.5.1.1
28 30
NARAK 11 11 11
2.5.1.2
7
8. decomposing visual scanning
similar to keystroke
decomposing: K, P, D, M ...
then predicting time
set visual tasks:
Memorizing information M
Entering and exiting (switching) representation
Seeking a subset of marks
Unpacking a mark and verifying a predicate
Seeking and navigating among a subset of marks
visualization «improvements» can be explained by these
visual tasks
8
9. «how to integrate a flight?»
M
1 2.1 TBO
AGN
270
CRX92
C
21
28 2.2
11
M 11
NARAK
M
North->South
even altitude AF121ZL 100 FISTO LMG
2.3
14 26
LFBO 11 11
AF497PY 120 TAN AUCHE AGN PERIG
South->North 2.3.2 15 2.3.1
12 18 28
LFBT 11 11 11 11
odd altitude
SWR686 290 AGN TBO SOVAR
19 28 30
NARAK 11 2.3.2.1
11 11
2.3.2.2
verifying a predicate: color helps matching similar flights
seeking and navigating: color selection (a la Bertin)
narrows set of flights to compare
9
10. «how to integrate a flight?»
TBO
M AGN
270
1 C
CRX92 21
28 11
11
NARAK
2.1
AF121ZL 100 FISTO LMG
2.2
14 26
LFBO 11 11
AF497PY 120 TAN AUCHE AGN PERIG
2.2.1
12 15 18 28
LFBT 11 11 11 11
2.3
SWR686 290 AGN TBO SOVAR
19 28 30
2.3.1
NARAK 11 11 11
seeking and navigating: color selection (a la Bertin)
narrows set of flights to compare
memorizing: only one memory cell (vs 3)
10
13. «how long will I wait?»
M M bus line
1 M current time 2 3
4
seeking and navigating among a subset of marks: times
of departure are displayed in a ordered manner
seeking a subset of marks: easy selection of elements to
the right of the element found in step 2 (selection
based on location).
memorizing: less information to memorize (3 vs 6).
…and there are no apparent drawback.
13
14. Dragicevic, P. and Huot, S. 2002. SpiraClock:
a continuous and non-intrusive display for
upcoming events. In Extended Abstracts of «how long will I wait?»
CHI '02. ACM, 604-605.
bus line 8h
M M 9h
10h
11h
12h
13h
14h
15h
16h
17h
1
18h
2
3
entering: current time directly visible (hands).
navigating: since the time is visible, navigating to the next
correct bus is shorter
exiting: rough idea of the waiting time directly visible (no
computation needed, culturally-known scale).
… and there are no apparent drawback. 14
15. «how long will I wait?»
M M bus line
1 M current time
2
navigating: linear layout + space between rows, visual
steering task easier
… at the expense of the entering operation (no current
time visible, since the representation is not dynamic).
15
16. summary
Munzner, T. A Nested Process Model for Visualization
Design and Validation. In IEEE Trans. on
Visualization and Computer Graphics, vol. 15, no. 6,
pp. 921-928, 2009.
threat: wrong problem deciphering a visualization is an
negatives and some false positives: many well-designed tools fail to
validate: observe and interview target users be adopted, and some poorly-designed tools win in the marketplace.
threat: bad data/operation abstraction
threat: ineffective encoding/interaction technique
overlooked problem
Nevertheless, the important aspect of this signal is that it reports what
the target users do of their own accord, as opposed to the approaches
validate: justify encoding/interaction design below where target users are implicitly or explicitly asked to use a tool.
threat: slow algorithm
validate: analyze computational complexity
identified a set of visual tasks that
3.4 Abstraction Threats
implement system
validate: measure system time/memory elicit efforts to be made
At the abstraction design level, the threat is that the chosen operations
and data types do not solve the characterized problems of the target
validate: qualitative/quantitative result image analysis
audience. The key aspect of validation against this threat is that the
[test on any users, informal usability study]
system must be tested by target users doing their own work, rather can be
improvement of designs
validate: lab study, measure human time/errors for operation
than an abstract operation specified by the designers of the study.
validate: test on target users, collect anecdotal evidence of utility
validate: field study, document human usage of deployed system
validate: observe adoption rates
explained with this set
A common downstream form of validation is to have a member of
the target user community try the tool, in hopes of collecting anecdotal
evidence that the tool is in fact useful. These anecdotes may have the
g. 2. Threats and validation in the nested model. Downstream levels shared vocabulary, design rationale
form of insights found or hypotheses confirmed. Of course, this obser-
vation cannot be made until after all three of the other levels have been
re distinguished from upstream ones with containment and color, as in fully addressed, after the algorithm designed at the innermost level is
gure 1. Many threats at the outer levels require downstream valida- implemented. Although this form of validation is usually qualitative,
on, which cannot be carried out until the inner levels within them are some influential work towards quantifying insight has been done [37].
ddressed, as shown by the red lines. Usually a single paper would only A more rigorous validation approach for this level is to observe and
ddress a subset of these levels, not all of them at once. document how the target audience uses the deployed system as part of
their real-world workflow, typically in the form of a longer-term field
study. We distinguish these field studies of deployed systems, which
ure. We use the word validation rather than evaluation to underscore are appropriate for this level, from the exploratory pre-design field
he idea that validation is required for every level, and extends beyond studies that investigate how users carry out their tasks before system
ser studies and ethnographic observation to include complexity anal- deployment that are appropriate for the characterization level above.
sis and benchmark timings. In software engineering, validation is We do echo the call of Shneiderman and Plaisant [39] for more field 16
17. open questions
is the set correct and complete ?
what facilitates/hinders visual task accomplishment ?
how to use it for design ?
how to analyze correctly and completly ?
predictive model: difficult...
Lohse, G. L. 1993. A cognitive model for
understanding graphical perception. Hum.-
Comput. Interact. 8, 4 (Dec. 1993), 353-388.
17
19. «how to integrate a flight?»
M
CRX92C 270 2.1
AGN
1 M
10 15 20
AF121ZL 100
FISTO
2.2
10 15 20
2.2.1
AF497PY 120
TAN AUCHE AGN
10
2.3 15 20
SWR686 290
AGN
10
2.3.1 15 20
+:
visual steering task, beacon search is facilitated (seeking and navigating)
Verifying a predicate: the time limit is directly visible.
-:
additionnal interaction to reach beacons not yet visible on the time scale.
19
Hence, colored strip holders enable controllers to narrow the set of flights to compare with a new one, and reduce the number of required steps accordingly (step 2.x, with x>=3, seeking and navigating). Holder colors can also ease predicate verification: holder color of the arriving strip can be matched easily to holder color of other strips, without requiring the controller to determine if the strip is a north-south or a south-north flight.
This facilitates seeking and navigating in step 2.x, as it reduces the subset of marks to consider when comparing times, and memorizing (1 vs 3 cells).
By the way, how easy is it to accomplish another task with the same visualization ?
Any visualization maximizes a task, but still, there are other tasks that we are supposed to do with a particular visualization
even if theoretical...
Compared to the regular strip boards, this design may aid…:
Seeking and navigating: thanks to a steering task, beacon search is facilitated.
Verifying a predicate: the time limit is directly visible.
… at the expense of a supplemental interaction to reach beacons not yet visible on the time scale.