Slides for the talk I gave at CSCW 2013, held in San Antonio, TX, USA. The full paper reference is: El Ali, A., van Sas, S. & Nack, F. (2013). Photographer Paths: Sequence Alignment of Geotagged Photos for Exploration-based Route Planning. In proceedings of the 16th ACM Conference on Computer Supported Cooperative Work and Social Computing (CSCW '13), 2013, San Antonio, Texas. Paper link: http://staff.science.uva.nl/~elali/pdfs/p985-el-ali.pdf

1. Photographer Paths: Sequence Alignment of Geotagged
Photos for Exploration-based Route Planning!
Feb.
26,
2013
Abdallah
‘Abdo’
El
Ali
Sicco
van
Sas
Frank
Nack
h6p://staff.science.uva.nl/~elali/

2. Outline!
I. Introduc3on
II. Photographer
Paths
III. User
Evalua3on
IV. Results
V. Discussion
&
Future
Work
2

3. Introduction
3

4. 4

5. 5

6. Pffttt…
6

7.
We
don’t
always
want
to
supply
user
preferences
7

8. Social
and
local
interpreta3on
of
city
places
and
routes
8

9. Off-­‐the-­‐beaten
track,
social
trails,
9
≠
Lonely
Planet!

10. Assump3on:
Loca3ons
of
photographs
are
poten3ally
interes3ng
10

11. But
sequen3al
property
needs
to
be
captured!
11

12. Sequence
Alignment
methods
12

13. Analysis
of
mobility
behavior
of
city
photographers:
 where
photographers
have
been
 in
what
order
they
have
been
there
 how
closely
their
movements
parallel
those
of
other
photographers
13 By Keiichi Matsuda via supercolossal

14. Research Questions!
How
can
walkable
route
plans
be
automaCcally
generated
for
residents
(and
tourists)
that
would
like
to
explore
a
city?
And
are
these
route
plans
desirable?
Three
factors:
1)
Which
data
sources?
2)
Which
methods
to
generate
routes?
3)
User
percep3ons
compared
to
fastest
and
popular
routes?
14

15. Photographer Paths!
15

16. Approach!
1)
Crawl
Flickr

geotags,
3mestamps
2)
Map
each
geotag/loca3on
in
a
sequence
to
a
cell
in
a
par33oned
grid
map
3)
Mul3ple
Sequence
Alignment
on
photographer
routes
to
find
aligned
loca3on
sequences

These
alignments
are
Photographer
Route
Segments
(PRSs)
16

17. Dataset!
 Flickr
geotagged
photos
within
Amsterdam,
The
Netherlands
 Area:
17.3
km
N-­‐S
and
24.7
km
E-­‐W
(center)
 5-­‐year
period
(Jan.
2006
-­‐
Dec.
2010)
 Aeributes:
 owner
ID
 photo
ID
 date
and
3me-­‐stamp
 la3tude
and
longitude
(street
level
accuracy)

Database:
426,372
photos
17

18. Preprocessing!
 Sequence
inference
with
following
constraints:
 photo
taken
within
4
hours
from
previous
photo
and
in
same
order
 minimum
2
or
more
different
loca3ons
(or
nodes)
 early
experiments
determined
125
x
125m
cells
in
center
of
Amsterdam
grid
suitable

1691
routes
(average
length
of
9.92
loca3ons)

1130
unique
photographers
18

19. Sequence Alignment!
 To
find
photographer
paths
from
Photographer
Route
Segments
(PRSs),
constraints
set:
 PRS
has
minimum
4
photographers
with
minimum
2
aligned
nodes/loca3ons
!#**% Photographer Route Segments (PRSs)!
!"#$%
!"**%
!+**%
%&'()*+$,*-.*)/*0$
!***%
$**%
##&% +%,-./.012,-314%

231
PRSs
#**% '!&% )%,-./.012,-314%
(average
length
of
2.61
nodes)
"%,-./.012,-314%
"**%
!"#$
+**% !'(%
!*!%
+'% !*% !% )'%
*%
+% )% "% '%
1)'-.*$&2/342)0$
19

20. PRSs in Amsterdam!
20

21. PRS Aggregation!
 Modified
Dijkstra’s
shortest
path
algorithm
&
P1N1 P2N1
&
# !
P1N2 !
" #
!"
#$%
Start P1N3 P2N2 End
21

22. PRS Aggregation to Crude Routes!
PRSs CM Photographer Route
WW Photographer Route
22

23. User Evaluation
23

24. Laboratory Study Design!
 ~45
min.
Quan3ta3ve/Qualita3ve
lab-­‐based
study
 15
par3cipants
(10
m,
5
f)
aged
between
21-­‐35
(M
=
29.2;
SD
=
3.3)
 Interac3ve
web-­‐based
prototype
route
planner
 Expert
route
evalua3on
by
‘city
residents’
(lived
in
Amsterdam
>
1
year)
 Plain
routes
to
avoid
informa3on
type
bias
24

25. Laboratory Study Design!
 Two
scenarios:
 Route
1:
Central
Sta3on
to
Museumplein

anernoon
scenario
favoring
explora3on
 Route
2:
Waterlooplein
to
Westerkerk

evening
scenario
favoring
efficiency
 Baseline
comparisons:
 Photo
Density
(PD)
route:
highest
density
of
photos
(over
5
year
period)
in
grid
cells
along
route
 Google
Maps
(GM)
route:
shortest
route
between
two
loca3ons
 Counterbalanced
within-­‐subject
design
 Route
Varia3on
(IV):
Photographer
Paths
vs.
Photo
Density
vs.
Google
Maps
25

26. Central Station to Museumplein (CM)!
Photographer Paths Photo Density Google Maps
route (5.36 km) route (3.83 km) route (3.35 km)
26

27. Waterlooplein to Westerkerk (CM)!
Photographer Paths
route (2.28 km)
Photo Density
route (2.60 km)
Google Maps
route (1.59 km)
27

28. Laboratory Study Design!
Data
collected:
1. AerakDiff2
(Hassenzahl,
2003)
UX
ques3onnaire
responses
[7-­‐point
seman3c
differen3al
scale]:
Usability,
Hedonic
Quali3es
(Iden3ty,
S3mula3on),
Aerack3veness
2. Two-­‐part
semi-­‐structured
interviews
Part
1:
Route
preferences,
feedback
on
Photographer
Paths
Part
2:
Inves3ga3on
of
visualized
informa3on
types
(visualized
info
type
handouts):
a)
Google
maps
b)
Color
coded
PRSs
(PP
route)
c)
Density
geopoints
(PD
route)
d)
Thumbnail
photo
geopoints
e)
Foursquare
POIs
28

29. a) d)
b)
e)
c)
29

30. Web Survey Study!
 Short
web-­‐based
survey
for
CM
and
WW
routes
and
varia3ons
 Basic
demographics
collected:
age,
gender,
years
in
Amsterdam
 Sta3c
route
images,
no
counterbalancing
 82
par3cipants
(55
m,
27
f)
aged
between
17-­‐62
(M=
27.6;
SD=
6.1)
 Most
lived
in
Amsterdam
for
more
than
3
years
(44/82)
 Some
between
1-­‐3
years
(15/82)
 Less
than
a
year
(11/82)
 Only
visited
before
(12/82)
30

31. Results
31

32. AttrakDiff2 !
Central Station to Museumplein (CM) Route
"#
** ** ** ** *
** ** *
$#
%#
!"#$%
&#
!%#
!$#
!"#
'()*+),-#./)0123#4'.5# 6789:1-#./)0123#!#;87:,23# 6789:1-#./)0123#!# =>()-,?7:7@@#4=AA5#
46.!;5# <,+/0),9:#46.!<5#
&'("$)'*$%
** !"#"$%$&
* !"#"$%$' 'B929*()CB7(#')2B@# 'B929#D7:@123# E99*07#F)C@#
32

33. AttrakDiff2 !
Waterlooplein to Westerkerk (WW) Route
** **
"# ** **
$#
%#
!"#$%
&#
!%#
!$#
!"#
'()*+),-#./)0123#4'.5# 6789:1-#./)0123#!#;87:,23# 6789:1-#./)0123#!# =>()-,?7:7@@#4=AA5#
46.!;5# <,+/0),9:#46.!<5#
&'("$)'*$%
!"#"$%$&
**
'B929*()CB7(#')2B@# 'B929#D7:@123# E99*07#F)C@#
33

34. Route Preference!
Lab
Study
 CM
route:
most
chose
to
follow
the
PP
route
(9/15),
PD
route
(4/15),
GM
route
(2/15)
 “One
of
the
routes
[PP]
was
long
and
took
many
detours,
and
I
thought
that
was
a
very
aFracHve
route!”
 WW
route:
most
chose
to
follow
GM
route
(10/15),
PD
route
(4/15),
no
route
(1/15)
 “You
are
going
for
coffee
so
you
just
want
to
get
there,
unlike
in
the
first
[CM]
scenario
where
it
is
a
nice
day
and
you
have
Hme.”
Web
Survey
•
CM
route:
GM
(40/82),
PD
(23/82),
PP
route
(10/82),
neither
(9/82)
•
No
experimenter
steering;
many
Amsterdam
residents
know
the
city
already
quite
well!
•
“I
would
not
easily
walk
these
routes...
who
in
Amsterdam
walks?
;)”
•
WW
route:
GM
(67/82),
PD
(6/82),
PP
route
(3/82),
neither
(6/82)
34

35. Digital Information Aids!
Lab
Study
Interview:
Part
1
 “How
many
persons
(focus
on
city
photographers)
took
a
given
route
segment
over
a
certain
3me
period
(e.g.,
1
year)?”
 Useful
(8/15)
for
exploring
a
city
one
already
knows
 Not
sure
(4/15)
 Depends
on
which
photographers
(2/15)
 Not
for
me
(1/15)
Interview:
Part
2
 Found
PP
info
type
aerac3ve
(10/15),
but
combine
with
Photo
thumbnails
(3/10)
and
POIs
(3/10)
35

36. Digital Information Aids!
Web
Survey
 POIs
along
a
route
(51x)
 Route
distance
(51x)
 Comments
along
a
route
(ranked
by
highest
ra3ngs
or
recency)
(24x)
 Expert
travel
guides
(22x)
 Photos
of
route
segments
(17x)
 No
digital
aids
(13x)
 Number
of
photographers
that
took
a
given
path
over
a
Hme
period
(9x)
 Number
of
photos
along
a
route
over
a
3me
period
(9x)
36

37. Discussion
37

38. Discussion!
 Discrepancy
between
lab-­‐study
and
web
survey
 Quick
web
survey
insufficient?
 Visualiza3on/explana3on
of
digital
aids
important?
 Proof-­‐of-­‐concept
approach
requires
real-­‐world
‘outdoor’
evalua3on
 Different
street
grid
network
 Scalability
to
larger
ci3es
 More
context-­‐awareness
38

39. Take Home Message!
Going
towards
data-­‐driven
explora3on-­‐based
route
planners…
 Some3mes
it’s
the
journey,
not
the
des3na3on
 A
quan3ta3ve
approach
may
oversimplify
human
needs
for
explora3on
 But
some3mes
we
want
an
automa3c
solu3on,
so
as
not
to
be
bothered
with
supplying
user
preferences
and
encounter
serendipity
39

40. Questions
40 h6p://staff.science.uva.nl/~elali/

41. References!
1.
Cheng,
A.-­‐J.,
Chen,
Y.-­‐Y.,
Huang,
Y.-­‐T.,
Hsu,
W.
H.,
and
Liao,
H.-­‐Y.
M.
Personalized
travel
recommenda3on
by
mining
people
aeributes
from
community-­‐contributed
photos.
In
Proc.
MM
’11,
ACM
(2011),
83–92.
2.
M.
Clements,
P.
Serdyukov,
A.
P.
de
Vries,
and
M.
J.
Reinders.
Using
flickr
geotags
to
predict
user
travel
behaviour.
In
Proc.
SIGIR
’10,
pages
851–852.
ACM
Press,
2010.
3.
M.
De
Choudhury,
M.
Feldman,
S.
Amer-­‐Yahia,
N.
Golbandi,
R.
Lempel,
and
C.
Yu.
Automa3c
construc3on
of
travel
i3neraries
using
social
breadcrumbs.
In
Proc.
HT
’10,
pages
35–44.
ACM
Press,
2010.
4.
F.
Girardin,
F.
Calabrese,
F.
D.
Fiore,
C.
Ra|,
and
J.
Blat.
Digital
footprin3ng:
Uncovering
tourists
with
user-­‐generated
content.
IEEE
Pervasive
Compu3ng,
7:36–43,
October
2008.
5.
N.
Shoval
and
M.
Isaacson.
Sequence
alignment
as
a
method
for
human
ac3vity
analysis
in
space
and
3me.
Annals
of
the
Associa3on
of
American
Geographers,
97(2):282–297,
2007.
6.
A.
Vaccari,
F.
Calabrese,
B.
Liu,
and
C.
Ra|.
Towards
the
socioscope:
an
informa3on
system
for
the
study
of
social
dynamics
through
digital
traces.
In
Proc.
GIS
’09,
pages
52–61.
ACM
Press,
2009.
7.
Hassenzahl,
M.,
Burmester,
M.,
and
Koller,
F.
AerakDiff:
Ein
Fragebogen
zur
Messung
wahrgenommener
hedonischer
und
pragma3scher
Qualit¨at.
Mensch
&
Computer
2003.
Interak3on
in
Bewegung
(2003),
187–196.
8.
Lu,
X.,
Wang,
C.,
Yang,
J.-­‐M.,
Pang,
Y.,
and
Zhang,
L.
Photo2trip:
genera3ng
travel
routes
from
geo-­‐tagged
photos
for
trip
planning.
In
MM
’10,
ACM
(2010),
143–152.
9.
Wilson,
C.
Ac3vity
paeerns
in
space
and
3me:
calcula3ng
representa3ve
hagerstrand
trajectories.
TransportaHon
35
(2008),
485–499.
41

42. Related Work!
 Sequence
Alignment
(SA)
methods:
 Borrowed
from
bioinforma3cs
and
later
3me
geography
 Time
geography
systema3cally
analyzes
and
explores
the
sequen3al
dimension
of
human
spa3al
and
temporal
ac3vity
(Shoval
&
Isaacson,
2007).
 Visualize
human
movement
on
2-­‐D
plane:
x-­‐
&
y-­‐
axis

longitude
and
la3tude;
z-­‐axis

3me

useful
for
analyzing
sequences
of
human
ac3vity
(in
this
case,
photo-­‐taking
behavior
of
photographers)
 Photo-­‐based
City
Modeling:
 Understand
tourist
site
aerac3veness
based
on
geotagged
photos
(Girardin
et
al.,
2008)
 Construct
inter-­‐city
travel
i3neraries
(De
Choudhury
et
al.,
2010)
 Generate
personalized
Point-­‐of-­‐Interest
(POI)
recommenda3ons
of
where
to
go
in
a
city
based
on
the
user's
travel
history
in
other
ci3es
(Clements
et
al.,
2010)

Approaches
focus
on
describing
loca3ons,
not
on
fine-­‐grained
within-­‐city
routes
that
connect
them
 Non-­‐efficiency
Driven
Route
Planners
 Automa3c
genera3on
of
travel
plans
based
on
millions
of
photos
(Lu
et
al.,
2010)
 Personalized
data-­‐driven
travel
route
recommenda3ons
(Cheng
et
al.
2011)

Systems
geared
towards
recommending
hotspots
and
popular
routes,
not
off-­‐beat
explora3on
42 routes

43. Sequence Alignment Overview!
Input:
two
sequences
over
the
same
alphabet
Output:
an
alignment
of
the
two
sequences
Example:

Source:
GCGCATGGATTGAGCGA
Target:
TGCGCCATTGATGACCA
 A
possible
alignment:
-­‐GCGC-­‐ATGGATTGAGCGA
TGCGCCATTGAT-­‐GACC-­‐A
Three
opera3ons
(each
with
cost):
 Perfect
matches
(MATCH)
 Mismatches
(DEL)
 Inser3ons
&
dele3ons
(INDEL)
 The
less
distance
cost,
the
higher
the
similarity
between
two
sequences
(Shoval & Isaacson, 2007)
43

44. Multiple Sequence Alignment Overview!
 Used
ClustalTXY
sonware
(Wilson
et
al.,
2008)
for
photo
alignment:
 makes
full
use
of
mul3ple
pairwise
sequence
alignments,
where
alignments
are
computed
for
similarity
in
parallel
 uses
a
progressive
heuris3c
to
apply
mul3ple
sequence
alignment
(MSA)
 allows
elements
to
be
represented
with
up
to
12-­‐character
words,
which
allows
unique
representa3on
of
small
map
regions,
used
for
represen3ng
the
geotagged
photos
 to
deal
with
differences
in
sequence
length,
ClustalTXY
adds
gap
openings
and
extensions
to
sequences.
 MSA
in
3
stages:
1)
Pairwise
alignments
are
computed
for
all
sequences
2)
Aligned
sequences
are
grouped
together
in
a
dendogram
based
on
similarity
3)
Dendogram
used
as
a
guide
for
mul3ple
alignment
44