1. NSF
Workshop
on
From
OpenSHAPA
to
Open
Data
Sharing
Arlington,
VA,
15-­‐16
Sep
2011
Suppor&ng
Emergence:
Interac&on
Design
for
Visual
Analy&cs
Approach
to
ESDA
William
Wong
Head,
Interac&on
Design
Center
Middlesex
University
London,
UK
15
September
2011
1

2. 2

3. What
we
do
in
ESDA
• Tool
usage
in
observa&on,
data
analysis
and
interpreta&on
– The
resolu,on
(wing
touch),
tool
differences
and
hence
what
can
be
done,
in
different
contexts
eg
development,
learning,
teaching
etc
• Sharing
of
collected
data
– Why
would
I
want
to
share
– If
I
could
share,
what
problems
and
hinderances
• Very
insighMul
of
the
specific
challenges
and
nuances
of
use
in
each
domain
of
use
• What
can
we
learn
from
a
different
form
of
“ESDA”
for
a
future
OpenSHAPA
/
OpenSHARE?
– From
security
and
library
domains
– Data
sharing
–
‘common
source’
but
used
by
different
analysts
– While
analysis
is
crucial,
sense-­‐making
to
draw
conclusions
based
on
assembled
evidence
for
making
decisions
is
paramount
– Use
Interac,ve
Visualisa,on
to
couple
intelligent
analysis
(e.g.
automa,c
en,ty
extrac,on,
automa,c
thema,c
analysis)
with
emergence
driven
user
interface
design
3

4. Learning
from
a
Security
and
Library
Perspec&ve
• Making
Sense
– EPSRC,
9
UK
Universi,es;
Imperial
College
PI,
MU
Deputy
PI
–
Mul,-­‐disciplinary
approach,
as
the
problem
cannot
be
addressed
by
a
single
discipline
e.g.
image
analysis,
corpus
linguis,cs
and
automated
en,ty
extrac,on,
soVware
forensics,
systems
engineering,
representa,on
design,
psychology,
law
• UKVAC
Phase
2
– US
DHS
and
UK
HM
Government,
5
UK
Universi,es,
Coordinator
MU
– Mul,-­‐disciplinary
approach
to
Nobel
Laureate
and
FAA
Flight
Data
Problem
• INVISQUE
– JISC
Rapid
Innova,on
Programme,
MU
PI
– Conceived
as
next
genera,on
alterna,ve
to
difficult-­‐to-­‐use
library
e-­‐resources
=>
tangible
reasoning
workspace
– Taylor
and
Francis
• Visual
Analy&cs
BLWWong©2011
4

5. What
is
Visual
Analy&cs?
• Visual
analy&cs
is
the
science
of
analy&cal
reasoning
facilitated
by
interac&ve
visual
interfaces
(Thomas
and
Cook,
2005).
– Integra,ng
tools
for
interac,ng
with
the
abstract
human
thinking
and
reasoning
processes
– Manipula,on
helps
in
reasoning
by
enabling
the
user
to
re-­‐arrange
the
problem
space
(Maglio
et
al,
1999)
• Data
graphics
or
info
vis
are
sta&c
• VA
combines
interac5ve
visualiza5ons
based
on
analy5c
tools
to
enable
rapid
querying
and
interroga&on
of
informa&on
…
– Visual
form
includes
charts,
network
graphs,
rela,onships
over
,me
and/
or
(geographical)
space
– enables
explora,on
through
rapid
and
repeated
querying
– access
to
original
data,
– analysis
of
data
– genera,on
of
hypotheses
– construc,on
of
conclusion
pathways
• …
for
the
purpose
of
sense-­‐making
– The
ability
to
rapidly
(and
visually)
process
and
assemble
evidence
to
enable
genera,on
of
explana,ons
or
conclusions,
enabling
decisions
5

6. The
Visual
Analy&cs
Problem:
Emergence,
Search
and
Explana&on
Visually
supported
analy,c
reasoning
Varied
media,
varied
analysis
and
presenta,on
tools
Frame
of
Reference
Lack
of
the
‘big
picture’
Keyhole
problem
Jig-­‐saw
puzzle
(not
one,
but
many)
Large
data
sets:
mul,-­‐sourced,
mixed-­‐format,
silo-­‐based,
sta,c/stream,
out
of
sequence,
uncertain
and
varying
quality
BLWWong©2011

7. 20
Representa&on
and
Analy&c
Problems
1. The
problem
of
seeing
a
large
data
set
and
reasoning
space
through
a
small
keyhole.
2. The
problem
of
handling
missing
data.
3. The
problem
of
handling
decep&ve
/
misleading
data.
4. The
problem
of
handling
contradictory
data.
5. The
problem
of
aggrega&ng
and
reconciling
mul&ple
points
of
view
or
predic&ons.
6. The
problem
of
evidence
colla&on
and
eviden&al
reasoning.
7. The
problem
of
provenance
and
tracing
analy&c
reasoning.
8. The
problem
of
integra&ng
data
space,
analy&c
space
and
hypothesis
spaces.
9. The
problem
of
handling
strength
of
evidence
(including
subjec&ve
and
objec&ve
measures
of
strength)
+
contribu&on
of
different
pieces
of
evidence
to
a
conclusion.
10. The
problem
of
handling
uncertainty
in
data
and
/
or
informa&on.
11. The
problem
of
represen&ng
and
handling
evidence
over
&me
and
space.
12. The
problem
of
annota&ng,
remembering,
re-­‐visi&ng,
and
sehng
aside.
13. The
problem
of
developing
a
sense
of
what
is
in
the
data
–
exploring
what
is
there.
14. The
problem
of
predic&ng
and
represen&ng
emergent
behaviour.
15. The
problem
of
Iden&fying
and
represen&ng
trends.
16. The
problem
of
recognising
and
represen&ng
anomalous
data.
17. The
problem
of
finding
the
needle
in
the
haystack
(or
knowing
what
is
chaff
–
i.e.
info
of
no
or
low
value)
18. The
problem
of
predic&ng
the
path
of
cascading
failures
or
effects.
19. The
problem
or
represen&ng
the
sta&c
and
dynamic
rela&onship
between
the
data
/
informa&on.
20. The
problem
of
scalability
and
reusability.
7
BLWWong©2011

8. Visual
Analy&cs
Concept
Interac,ve
Dynamic
querying
Visualiza,on
of
Output
Palerns
and
commonali,es
Filters
Seman,c
Extrac,on
Data
integra,on
&
transforma,on
Many
tools
Sensors
/
Surveillance
/
Data
collec,on
“SoV”
Data
“Hard”
Data
Social
networks,
interac,ons,
ac,vi,es
BLWWong(c)2010
8

9. Architecture:
Many
Tools

10. Indexing,
Structuring
and
Theorizing:
Visual
Analy&cs
and
OpenSHAPA
Indexing
Structuring
Theorizing
Data
Sets
Automated
Schema,za,on
Explana,ons
-­‐ Structured
en,ty
extrac,on
Search
and
query
Hypothesis
and
Analy,cal
tools
tes,ng
unstructured
Colla,on
for
topical,
Eviden,al
text
geospa,al,
Thema,c
analysis
-­‐ Video
reasoning
temporal,
-­‐ Speech
Conclusion
network
analysis
Not
just
reports
pathways
and
video,
but
also
social
media
Provenance
–
data,
processes,
and
reasoning:
Traceability,
how
did
we
get
here?
10

11. Emergent
Themes
Analysis
Representation
Broad Themes Design Concepts
Related excerpts from transcripts
e.g. Goals
Decision
Strategies
Transcripts Interpret &
Conceptualise
e.g. Planning Specific themes
Excerpts relating to specific Narratives
concepts in a theme, e.g. types
of activities, examples of cues
e.g. Assessment of
Resources Activities Cues Knowledge Difficulties
Identify,
Index &
e.g. Assessment
Collate
e.g. Control
e.g. Assessment of
Structuring &
Situation
Data
reduction
Wong©2004
11

12. INVISQUE
demo:
Interac&on
Design
for
Suppor&ng
Emergence
• INterac&ve
VIsual
Search
and
QUery
Environment
– Visual
forms
alempt
to
create
palerns
that
reinforce
relaIonships
(CSE)
– Interac,on
designed
to
support
emergence
in
themaIc
analysis
• INVISQUE
JISC
Library
Version
– Suppor,ng
sense-­‐making
–
Data-­‐Frame
Model
– Using
the
basic
interac,ve
visualiza,on
techniques
developed
here
to
support
sense-­‐making
in
inves,ga,ve
domains
12

13. The
Interac&ve
Visualiza&on
Approach
• Informa&on
Design
Principles
– Focus+Context
– Proximity-­‐Compa,bility
Principle
– Gestalt
Principles
of
Form
Percep,on
– Principle
of
Visual
Affordances
– Ecological
Interface
Design
– Representa,on
of
Func,onal
Rela,onships
13

14. The
Interac&ve
Visualiza&on
Approach
• Principles
implemented
in
design
by
– Anima,on,
transparency,
informa,on
layering,
spa,al
layout,
palern
crea,on
– Emphasizing
the
representa,on
of
rela,onships
within
the
data
– Discovery
of
expected
and
un-­‐an,cipated
rela,onships
– Interac,on
techniques
enable
rapid
and
con,nuous
itera,ve
querying
and
searching
while
keeping
visible
the
context
of
search
– Minimizing
WWILF-­‐ing,
or
the
‘What
Was
I
Looking
For?’
problem
14

15. The
Data-­‐Frame
Model
Guides
Interac&on
Design
Klein
et
al,
2007
15

16. Reasoning
workspace
framework:
Mapping
and
design
and
of
reasoning
work
to
the
“keyhole”
Hypothesis
Space
Depic,on
of
-­‐ Collate,
assemble,
marshal
“reasoning
and
-­‐ Formula,on
search
process”
-­‐ Tes,ng
and
simula,on
-­‐ arguments,
conclusions,
“brushing”
evidence
Depic,on
of
“Data
terrain”
Conclusion
Pathways
Data
Space
Analysis
Space
-­‐ Tools
and
algorithms
-­‐ what’s
available?
-­‐ Behaviours,
rela,onships
-­‐ What’s
changed?
and
palerns
-­‐ Awareness:
what’s
in
there?
-­‐ what’s
going
on
in
there?
Transla&on
into
Design
BLWWong(c)2010
16

17. Conclusion:
Some
Ques&ons
• What
can
we
do
for
a
future
OpenSHAPA
and
OpenSHARE?
– indexing,
structuring,
bearing
in
mind
future
will
have
lots
of
“smart”
analysis
technologies
that
can
support
the
lower
levels
of
analysis,
par,cularly
indexing
• What
System
Architecture?
– that
combines
data
from
different
sources,
and
allows
a
variety
of
tools
to
analyse
and
make
sense
of
data
• Alterna&ve
designs
for
structuring
and
theorizing
that
more
directly
support
sense-­‐making?
– Adopt
/
adapt
an
interac,ve
visualisa,on
interface
design
– Focus
on
emergence,
search
and
sense-­‐making
• Emergence
techniques
such
as
“Temporal
narra,ves”
• Mul,ple
threads
/
parallel
lines
of
enquiry
and
finding
intersec,ng
storylines
– Reasoning
workspace
for
assembling
our
thoughts
and
conclusions
• Future
work:
Collabora&ve
Sense-­‐making
environments
17

18. End
18