The Songs of Our Past

THE SONGS
OF OUR PAST

DOMINIKUS BAUR
WORKING WITH
LISTENING HISTORIES UNIVERSITY OF
MUNICH (LMU)
GERMANY

Hi,
I’m
Dominikus
from
the
University
of
Munich.
I’m
a
fourth
year
Ph.D.
student
and
today
I
will

talk
about
some
of
the
work
that
I’ve
done
so
far.

In this talk:

As
you’ve
probably
already
guessed
from
the
Btle,
my
focus
is
on
a
special
type
of
personal

histories,
namely
music
listening
histories.
In
this
talk
I
will
ﬁrst
describe
what
listening
histories

are
and
what
we
mean
by
that.
Then
I’ll
show
you
some
projects
from
the
area
of
informaBon

visualizaBon
where
we
worked
with
listening
histories
from
single
or
mulBple
people
and
tried

to
make
them
understandable
for
them.
Finally
I
will
give
you
some
ideas
what
else
than

visualizing
we
could
do
with
this
type
of
data.

Photos,
be
they
analog
or
digital,
are
a
common
way
to
remember
the
past.
We
all
take
photos

while
on
vacaBon,
having
friends
over
or
for
all
these
other
occasions
and
aIerwards
look
at

them
(or
don’t)
and
think
about
the
past.
But
what
we
do
in
our
lives
is
oIenBmes
so
much

more
than
a
photo
can
capture

[click]
unfortunately
all
auditory
informaBon
is
lost
in
the
process.
The
music
we
made,
the

songs
we
heard.
Nowadays,
of
course,
there
is
oIen
a
structural
diﬀerence
between
both

acBviBes:
The
Bme
we
spend
acBvely
making
music
is
signiﬁcantly
smaller
than
the
Bme
we

spend
listening
to
it.

But
even
though
we
no
longer
make
the
music,
it
is
more
abundant
than
ever
before
thanks
to

our
mobile
gadgets.
And
so
these
songs
by
people
we
don’t
know
sBll
stand
for
parts
of
our

lives:

The
song
you
heard
during
this
one
summer…

Or
the
one
that
was
playing
when
you
met
a
special
someone…

…
and
of
course
the
songs
we
hear
every
year
for
special
occasions.

REMINISCING

So,
an
account
of
all
the
music
we
listened
to,
a
listening
history,
can
serve
for
reminiscing
just

as
well
as
photos.
In
this
regard,
listening
histories
are
a
part
of
the
so-‐called
lifelogging
data.

Lifelog

A digital representation of all
aspects of one’s life

Lifelogs
are
digital
representaBons
of
aspects
of
one’s
life.
So,
via
this
deﬁniBon,
every
facebook

status
and
blog
entry
already
stands
as
a
part
of
lifelog
data.
But
the
original
vision
of
lifelogging

consists
of
capturing
really
everything
that
you
experience.
And
the
original
visionaries
went
…

…
to
great
lengths
to
reach
that
goal.
So
while
capturing
listening
histories
is
only
a
humble

secBon
of
a
complete
lifelog,
they
can
sBll
bring
many
of
the
beneﬁts.

REMINISCING

Sellen, Whittaker: Beyond Total Capture: A
Constructive Critique of Lifelogging,
CACM, May 2010
In
a
recent
paper,
Abigail
Sellen
and
Steve
WhiXaker
idenBﬁed
some
of
the
beneﬁts
that

lifelogging
data
can
bring
and
summarized
them
as
the
‘5
Rs’.
We’ve
already
seen
reminiscing,

as
re-‐living
the
past
for
emoBonal
reasons.

RECOLLECTING
REMINISCING
RETRIEVING
REFLECTING
REMEMBERING Sellen, Whittaker: Beyond Total Capture: A
CACM, May 2010
But
there’s
more.

RECOLLECTING
REMINISCING
RETRIEVING
REFLECTING
CACM, May 2010
RecollecBng
is
the
more
general
(and
less
emoBonal)
case
of
reminiscing
and
can,
for
example,

mean
using
a
listening
history
to
ﬁnd
a
song
whose
name
I
have
forgoXen.

RECOLLECTING
REMINISCING
RETRIEVING
REFLECTING
CACM, May 2010
Retrieving
is
more
appropriate
for
text-‐
and
other
documents,
but
it
can
also
mean
that
I
can

immediately
listen
to
that
song.

RECOLLECTING
REMINISCING
RETRIEVING
REFLECTING
CACM, May 2010
ReﬂecBng
describes
the
process
of
thinking
about
your
life
using
the
lifelog.
Say,
I
listened
to
a

fair
share
of
pop
and
rock,
now
it’s
Bme
to
become
serious
and
listen
to
classical
music.

RECOLLECTING
REMINISCING
RETRIEVING
REFLECTING
CACM, May 2010
And
ﬁnally,
remembering
intenBons
describes
thinking
about
prospecBve
acBviBes,
such
as

regularly
checking
if
a
band
has
a
new
album
or
is
on
tour.

RECOLLECTING
REMINISCING
RETRIEVING
REFLECTING
CACM, May 2010
So,
these
‘ﬁve
Rs’
provide
a
good
overview
of
the
possible
beneﬁts
of
capturing
listening

histories.

Let
me
talk
a
liXle
bit
about
what
listening
histories
are
and
where
they
come
from.

Listening history

A complete chronological
collection of musical items
…

In
my
understanding
an
ideal
listening
history
describes
all
songs
that
a
person
has
listened
to,

possibly
in
their
lifeBme.
What’s
important
here
is
that
…

...
Each song:
(1) pre-existing piece of music
...

Each
song
is
a
pre-‐exisBng
piece
of
music
that
has
aXributes
such
as
arBst,
Btle,
etc.

…
(2) has been heard at least partially

And
second,
each
song
has
been
heard
by
the
owner
of
the
history
at
least
in
parts.
So,
the

quesBon
is,
where
do
we
get
such
data
from?

Fortunately,
there’s
a
popular
service
called
‘last.fm’
that’s
been
around
for
a
while
and
does

exactly
that.
Last.fm’s
actual
intenBon
for
capturing
a
person’s
listening
behavior
is
providing

beXer
recommendaBons
for
their
webradio,
but
the
resulBng
listening
histories
are
easily

accessible
through
their
API
which
makes
them
a
perfect
target
for
all
kinds
of
projects.

Last.fm’s
tracking
technology
is
called
‘Audioscrobbler’,
which
is
both
a
protocol
and
a
soIware.

Devices
and
media
players
can
either
use
the
protocol
directly
or
rely
on
the
background

audioscrobbler
process
running
on
the
user’s
machine.
And
in
the
end
we
arrive
at
a

chronological
list
of
/all/
the
songs
a
person
has
listened
to...

+
=
But
while
this
could
be
used
to
hypotheBcally
capture
the
complete
listening
history
of
a
person

and
works
great
in
theory
[click]

Reality
oIen
looks
a
bit
diﬀerent.

Real listening
histories:

- incomplete
- noisy

The
actual
resulBng
listening
histories
are
both
incomplete
and
noisy.
Let
me
just
tell
you
what
I

mean
by
that.

Real listening
histories:

- incomplete
- noisy

Gaps
in
a
listening
history
can
come
from
various
places…

One
common
source
is
that
the
listener
is
using
non-‐supported
hardware
for
listening

Another
that
music
comes
from
other
sources
like
when
shopping
or
being
at
a
friend’s
place.

Real listening
histories:

- incomplete
- noisy

Noise,
i.e.,
too
many
songs
are
tracked
is
also
quite
common…

The
user
might
leave
the
computer
while
the
music
keeps
on
playing…

Or
someone
else
is
using
the
computer
while
the
audioscrobbler
is
sBll
running.

> 50%

Another Caveat: The audioscrobbler only tracks a song after the user has listened to at
least half of it. Again, keep in mind that the main incentive for last.fm to track listening
is improving the recommendations of their web radio: If a song is skipped then the
listener probably didn’t like it and it’s uninteresting for recommendations.

30 million users / month
(March 2009)

http://blog.last.fm/2009/03/24/lastfm-radio-announcement

SBll,
despite
these
downsides,
last.fm’s
data
is
preXy
reliable
and
the
service
is
very
popular.

According
to
them,
30
million
people
visit
the
webpage
per
month.

In
the
end
we
arrive
at
a
chronological
list
of
songs
and
that’s
all
we
get.
Each
secBon
of
Bme

either
contains
music
or
it
does
not.
So
we
have,
for
example,
no
informaBon
on
the
context
of

the
music
listening
(I’ll
get
back
to
that
aspect
later).
SBll,
to
make
it
easier
to
understand
this

data
we
can
then
start
to
analyze
it
and
e.g.,
to
extract
listening
sessions.

Listening
sessions
are
characterized
by
the
gaps
between
the
songs,
so
a
gap
of
e.g.,
half
an

hour
between
two
songs
means
that
the
creator
of
the
history
stopped
listening
and
thus

ended
the
session.
To
make
these
histories
a
bit
more
meaningful
we
can
also
go
beyond
the

single
Bme
dimension…

Genre
……

Sub-Genre

Artists

Albums

Songs

…
and
put
the
songs
into
the
musical
hierarchy
of
albums,
arBsts
and
genres.
While
this

classiﬁcaBon
is
not
perfect
and
oIenBmes
the
topic
of
heated
debates,
at
least
it’s
widely-‐
known
among
all
music
listeners.
One
more
step
to
overcome
the
downsides
of
a
strict

hierarchy
is
adding
user-‐generated
keywords
into
the
mix…

Genre
……

Sub-Genre

Artists

Albums

Songs

Tags
…
that
can
become
a
stand-‐in
for
any
number
of
diﬀerent
hierarchies
or
classiﬁcaBons.
You
will

see
some
of
these
aspects
in
the
prototypes
that
I’m
about
to
show
you.

But
back
to
the
actual
beneﬁts
to
the
creators
of
such
listening
histories,
think
of
the
‘5
Rs’
of

reminiscing,
recollecBng
and
so
on.
Here
you
can
see
the
default
view
of
Last.fm
presenBng
this

data:
A
chronological
web-‐based
list
which
is
not
that
helpful
for
any
of
these
tasks.
And
as

you’ve
just
seen,
listening
histories
can
become
quite
complex
once
you
dive
into
their
depths

which
makes
other
forms
of
presentaBon
more
useful.

As
a
ﬁrst
step
towards
understanding
this
data
and
also
making
their
owners
understand
them
I

started
building
visualizaBons
based
on
it.
To
make
maXers
not
overly
complicated,
…

…
I
used
only
a
single
listening
history,
i.e.
a
possibly
long
list
of
possibly
repeated
songs.

My
ﬁrst
approach
to
visualizing
this
type
of
data
was
a
node-‐link
diagram:
The
idea
was
that

each
unique
song
would
be
represented
as
a
node
…

…
while
each
pair
of
consecuBve
songs
would
form
an
edge
in
the
diagram.
And
while
this

concept
was
easy
to
understand,
the
result
wasn’t
–
necessarily.
And
you
might
also
understand

why
I
Btled
this
visualizaBon
‘ Tangle’:

Here
be
a
chaoBc
screenshot
of
tangle

Tangle
While
it
certainly
looks
chaoBc,
there
are
sBll
several
aspects
that
you
can
draw
from
it:

Here
be
a
chaoBc
screenshot
of
tangle

Tangle
For
one,
the
layout
of
the
nodes
is
force-‐directed,
which
means
that
nodes
with
many
edges

(i.e.
songs
that
appear
repeatedly
within
the
history)
are
drawn
towards
the
center,
…

Here
be
a
chaoBc
screenshot
of
tangle

Tangle
…
while
less
popular
songs
and
one-‐hit-‐wonders
are
on
the
outskirts.

Here
be
a
chaoBc
screenshot
of
tangle

Tangle
An
addiBonal
encoding
is
the
thickness
of
the
connecBng
arrows
that
represents
the
number
of

Bmes
this
two-‐song-‐sequence
was
played
which
shows
albums
and
pre-‐deﬁned
playlists.

VIDEO

Tangle
[VIDEO]
And
ﬁnally,
Tangle’s
layout
is,
as
I
said,
force-‐directed
which
means
that
the
user
is
able

to
interacBvely
explore
the
visualizaBon.
Zooming
and
panning
is
of
course
possible.
By

hovering
over
a
song
addiBonal
informaBon
is
shown.
And
the
user
can
drag
around
songs
at

will.

As
it
was
not
easy
to
learn
much
from
the
Tangle
visualizaBon,
I
wanted
to
put
some
sense
into

it.
Filtering
or
splihng
the
data
seemed
promising,
so
I
focused
on
listening
sessions
this
Bme.

The
basic
idea
was
again
a
node-‐link
diagram,
but
this
Bme
songs
could
appear
more
than
once.

This
Bme
the
more
important
factors
were
the
Bme
stamps
of
the
songs.
A
pause
of
in
this
case

1
hour
indicates
the
start
of
a
new
listening
session.

Strings
By
sorBng
the
sessions
chronologically
we
arrived
at
this
visualizaBon,
called
‘Strings’.

Strings
Zooming
out
gives
you
an
overview
of
the
length
of
your
listening
sessions,
shows
outliers
and

Bmes
when
you
didn’t
listen
to
music.
The
verBcal
Bme
line
is
very
important
in
this
regard.

Strings

Finally,
you
probably
wondered
about
the
blue-‐ish
arcs:
The
problem
with
Strings
is
that
each

song
can
possibly
appear
several
Bmes
in
the
visualizaBon
as
single
songs
are
no
longer

represented
by
single
nodes.
Therefore,
we
draw
arcs
between
idenBcal
songs
which
makes
it

possible
to
gauge
the
importance
of
one
song
or
see
repeBBve
sequences
(at
the
boXom).

?
So,
what
these
two
examples
had
in
common
that
they
were
both
restricted
visualizaBons
that

(1)
focussed
on
one
aspect
of
the
data
and
(2)
allowed
only
liXle
interacBon.

playful
If
you
want
to
put
a
label
on
them
it
would
probably
be
‘playful’
which
means:
They
are

designed
for
one
speciﬁc
aspect
of
the
data
which
cannot
be
customized.
They’re
built
for
this

task
only.
But
they
can
sBll
engage
the
user
to
play
around
and
interact
(at
least
a
liXle).

playful casual expert

If
you
want
to
put
this
into
an
infovis
perspecBve,
two
other
commonly
used
terms
are
useful:

‘Casual’
describes
visualizaBons
that
are
a
liXle
more
interacBve
and
customizable
but
not
as

complex
as
‘expert’
systems
that
allow
ﬁne-‐grained
customizaBon
but
require
solid
knowledge

in
the
respecBve
area.


For
visualizaBons:
A
type
of
playful
visualizaBon
would
be
Wordle,
engaging
but
with
a
single

purpose.
The
Many
Eyes
project
is
easy
to
use
but
has
much
more
ways
to
display
and
filter
the

data.
Finally,
programming
frameworks
such
as
protovis
or
processing
allow
utmost
flexibility

but
are
difficult
to
get
into
and
master.

Interactivity
expert

casual

playful
Complexity

If
we’re
inclined
to
put
these
three
concepts
into
relaBon
to
each
other,
we
can
use
interacBvity

and
complexity.
So,
playful
tools
aren’t
very
ﬂexible,
but
also
not
very
complex.
Expert
tools

however
are
mulB-‐purpose
and
highly
interacBve
but
also
diﬃcult
to
master.
It
depends
on
the

user
populaBon
and
the
task
what
visualizaBon
concept
to
choose.

Interactivity
expert

casual

playful
Complexity

In
our
case
with
listening
histories,
we
have
people
who
like
to
listen
to
music
and
are
not

necessarily
infovis-‐experts.
Also,
analyzing
their
listening
behavior
is
something
they
don’t
do

regularly
so
forcing
them
to
learn
something
for
using
a
complex
visualizaBon
will
rather
put

them
oﬀ
than
engage
them.
Therefore
I
concentrated
on
the
playful/casual
corner
of
this
design

space.

Ok,
so
back
to
the
visualizaBon.
Both
Strings
&
Tangle
were
very
single
purpose
and
liXle

customizable.
For
the
next
project,
I
wanted
to
give
users
more
freedom
in
analyzing
their

listening
histories
but
sBll
keep
the
tool
accessible.
Strings
&
Tangle
were
also
only
informally

evaluated
with
a
few
people
from
our
lab
so
I
wanted
to
see
if
real
people
would
actually
ﬁnd

something
like
that
useful…

LastHistory
...
The
result
was
LastHistory,
a
/casual/
infovis
tool
for
analyzing
and
reminiscing
in
one’s
own

listening
history.
We
made
it
available
on
the
internet.
Several
thousand
people
downloaded
it

and
we
received
lots
of
feedback.
When
designing
LastHistory
we
ﬁrst
wanted
to
make
sure
that

it
felt
easily
accessible
for
people.
The
visualizaBon
in
its
non-‐interacBve
state
should
already

give
insights
to
the
user,
and
so
gradually
lure
them
into
exploring
the
more
sophisBcated

opBons.

LastHistory
So,
the
largest
part
of
the
applicaBon
is
taken
up
with
a
2D
Bmeline:
all
songs
are
represented

as
small
circles
and
mapped
horizontally
to
the
day
and
verBcally
to
the
Bme
of
day
of
their

Bmestamps.
This
way,
users
can
easily
see
daily
rhythms,

LastHistory
Like
at
what
Bme
this
person
usually
went
to
bed.

LastHistory
And
here’s
another
example:
A
user
who
gets
up
at
the
same
Bme
everyday
and
listens
to
music

ﬁrst
thing
in
the
morning.

classical

jazz

funk

hip-hop

electronic

rock

metal

unknown/other

LastHistory
Each
song’s
genre
is
color-‐coded,
so
the
user
gets
an
immediate
overview
over
the
variety
of

songs.
We’re
of
course
restricted
in
the
number
of
colors
we
can
use
to
keep
them

disBnguishable.

LastHistory
Beyond
staBc
visualizaBon,
users
can
navigate
within
the
visualizaBon
by
panning,
triggered
by

dragging
with
the
mouse

LastHistory
One-‐dimensional
zooming
by
using
the
mouse’s
zoom
wheel
or
the
slider
in
the
lower
right

corner
allows
them
to
focus
on
certain
secBons
of
the
history.

LastHistory
Hovering
over
a
song
shows
a
box
with
user-‐generated
keywords
from
last.fm,
but
more

prominently:
connects
this
song
with
all
other
instances
of
it
throughout
the
history.
So,
users

can
easily
see
when
they
listened
to
this
one
song.

LastHistory
Preceding
and
succeeding
repeated
songs
are
also
highlighted,
so
sequences
such
as
albums
or

other
predeﬁned
playlists
are
automaBcally
highlighted.

LastHistory
Finally,
in
the
upper
right
corner
of
the
applicaBon,
there’s
a
textbox
for
ﬁltering
where
users

can
enter
freeform
terms.
It’s
possible
to
enter
song
or
album
Btles
or
arBst
names
to
ﬁlter
all

other
songs.

LastHistory
But
the
ﬁlter
box
can
also
be
used
for
temporal
queries
by
entering
dates,
or
periods
of
Bme,
so

users
can,
for
example,
see
all
songs
that
they
listened
to
in
autumn
before
noon.
But
enough

with
the
default
infovis-‐features.
One
interesBng
aspect
of
this
project
was
that
we
could
use
an

addiBonal
data
source
for
gaining
insights:
The
user’s
memories.

To
access
these,
we
needed
memory
triggers.
Some
research
in
psychology
has
shown
that

personally
created
things
such
as
photos
can
be
useful
in
this
regard,
so
we
integrated
photos

and
calendar
entries
from
the
user’s
harddisk.

Two usage modes:

Analysis Personal

We
split
these
into
two
diﬀerent
usage
modes
and
called
them
‘analysis’
(everybody
can
do
it)

and
‘personal’
(with
memory
triggers
that
probably
are
only
useful
to
the
owner
of
the
history).

So
in
this
personal
mode
we
have
contextual
informaBon
that
makes
it
easier
to
remember

what
happened
at
what
Bme
and
understanding
the
listening
decisions.
Users
could
simply

switch
between
the
two
modes
with
the
buXon
in
the
upper
leI
corner.

Ok,
so
much
for
the
tool.
As
I
said,
we
made
it
available
on
the
internet
and
a
lot
of
people

downloaded
it.

Praise on tech blogs

5,000 downloads

243 filled-out questionnaires

Some
numbers:
First
we
got
a
good
amount
of
coverage
on
tech
blogs,
which
led
to
a
certain

popularity.
Right
now,
we
have
about
5,000
downloads.
We
also
included
a
link
to
a

quesBonnaire
that
pops
up
aIer
fiIeen
minutes
of
using
the
tool
and
around
250
people

answered
that
quesBonnaire.
We
kept
that
intenBonally
short,
in
order
not
to
put
off
people
as

a
short
answer
is
beXer
than
no
answer
at
all.

About the Personal Mode:

“I like this mode the best, it
should be the default mode!”

“Clicking on a photo gallery and
listening to what I was listening
to at the time was very powerful.”

People
who
had
photos
and
calendar
entries
available
enjoyed
using
the
personal
mode
and

also
features
like
the
possibility
to
create
a
slideshow
of
music
and
photos.

About the Analysis Mode:

“I rarely listen to music between
the hours of 9-11 a.m., even on
weekends.”

“I noted the … commuting pattern.”

“Those ruts where you get stuck in
listening to one particular song.”

“I listened to music for 4 straight
days!”

And
in
general,
people
were
also
able
to
ﬁnd
repeaBng
paXerns
and
liked
how
they
were
able

to
learn
interesBng
aspects
about
themselves.

75% found it easy to use and learn.

Another
thing
we
learned
that
worked
really
well
was
puhng
a
ﬁve
minute
video
online
that

explained
how
to
use
the
tool:
There
was
no
online
help
or
something
like
that
available
and

sBll
75%
percent
found
it
easy
to
learn
and
use.

Finally,
people
really
liked
to
share
the
results.
And
as
there
was
no
straighporward
way
to
do
it

within
the
applicaBon,
they
resorted
to
taking
screenshots
for
posBng
it
on
ﬂickr
or
their
blogs.

So
a
future
version
should
deﬁnitely
take
that
into
account.

A

Ok,
so
these
were
three
examples
for
visualizing
single
listening
histories.
But
it
gets
much
more

interesBng
when
we
have
not
one
history…

E

D

C

B

A

…
but
many
more.
And
as
music
has
an
intricate
social
funcBon
as
well,
comparing
one’s
taste
in

music
to
friends,
family
and
peers
can
be
an
interesBng
use
case.

So
in
the
following,
I
will
give
you
two
examples
for
approaches
to
visualize
these
data.
Again,

I’ll
present
one
playful
and
one
casual
approach.

B

A

While
mulBple
histories
can
mean
a
lot
of
histories,
for
a
ﬁrst
approach,
I
decided
to
focus
on

just
two
histories.
One
will
usually
be
the
user’s
history
and
the
other
one
of
a
friend
or
another

person
that
he
or
she
knows.

Ok,
so
what’s
the
best
way
to
do
that:
Aligning
the
songs
to
a
Bme-‐line
is
probably
a
good
idea,

to
allow
comparisons
for
the
number
of
songs,
regularity
of
listening
and
so
on.
But
users
are

especially
in
this
one-‐on-‐one
scenario
interested
in
also
comparing
their
taste:
Are
both
of
them

listening
to
the
same
songs
or
arBsts?
Or
is
there
no
similarity?

An
easy
way
to
encode
that
is
using
the
distance
between
the
songs
from
each
history:
The

closer
a
song
gets
to
the
other
history,
the
more
similar
it
is
to
it,
resulBng
in
a
fever
chart
of

relatedness.

LoomFM
Here’s
an
example
from
the
resulBng
‘LoomFM’
visualizaBon.
You
have
a
horizontal
Bmeline
and

two
listening
histories
from
user
red
and
purple.
The
closer
one
of
the
small
song
circles
comes

to
the
Bmeline,
the
more
related
it
is
to
the
other
user’s
taste
in
music.

LoomFM
Some
more
things:
The
more
consecuBve
songs
share
the
same
genre
or
arBst,
the
larger
the

corresponding
label
gets.
By
doing
this,
important
arBsts
are
visible
even
when
zoomed
out.

Also,
labels
that
both
users
share
at
the
same
point
in
history
move
to
the
center
of
the

Bmeline.

LoomFM
AddiBonally,
the
yellow
arcs
connect
idenBcal
songs
–
the
same
principle
as
in
the
‘Strings’

visualizaBon.
Using
this
approach,
you
can
get
a
sense
if
a
person
repeatedly
listens
to
the
same

songs
(as
user
red
in
this
example)
or
only
once.
Also,
songs
that
both
users
share
are

connected…

LoomFM
…
as
in
this
example,
where
a
new
album
of
‘ Trail
of
Dead’
was
released
and
both
users

gradually
started
listening
to
it.
(you
can
also
clearly
see
the
sequence
here)

VIDEO

LoomFM
Here
you
can
see
a
video
of
LoomFM.
As
always,
zooming
and
panning
are
possible
and
gehng

more
informaBon
by
hovering
over
songs
or
arcs.


LoomFM
consBtutes
an
example
of
a
playful
visualizaBon
for
mulBple
histories.
The
tasks
are

clearly
deﬁned,
interacBon
is
minimal
and
a
lot
of
informaBon
(e.g.
the
similarity
between

songs)
is
implicit
and
predeﬁned…

Screenshot
Of
LastLoop


…
to
overcome
the
restricBons
and
also
to
integrate
more
than
two
histories
we
did
another

project
called
LastLoop
and
aimed
more
for
the
casual
area.

The
basic
idea
was
to
have
a
cross
between
LastHistory
and
LoomFM,
to
give
users
the
chance

to
do
these
more
complex
analyses
using
ﬁltering
and
things
like
that
while
also
being
able
to

connect
the
diﬀerent
listening
histories
and
see
relaBons
between
them.

LastLoop
Here’s
the
result
that
we
called
‘LastLoop’.
What
you
can
see
here
are
three
listening
histories

(you
can
have
an
unlimited
number
of
verBcally
stacked
histories),
arranged
to
the
same

Bmeline.

LastLoop
We
used
the
2D
Bmeline
metaphor
from
LastHistory
once
more,
so
it’s
possible
to
see
daily

paXerns
across
all
histories.

LastLoop
Also,
by
hovering
above
a
song,
all
other
occurences
within
this
one
history
and
the
others
are

highlighted
(re-‐using
the
metaphor
from
the
other
projects).

LastLoop
The
user
can
also
select
a
whole
area
and
again,
see
where
else
the
songs
appear.

LastLoop
Finally,
to
make
the
informaBon
manageable,
users
can
also
search
for
songs,
arBsts,
albums

and
so
on…

LastLoop
…
or
ﬁlter
for
certain
genres.

VIDEO

LastLoop
And
here’s
the
system
in
acBon:
You
can
pan
and
zoom
either
by
using
the
mouse
or
the
Bme

slider
at
the
boXom
of
the
screen,
select
screen
regions
to
see
other
occurences
of
the
selected

songs
(and
switch
between
all,
songs
from
the
selecBon
or
songs
within
the
selecBon
only).

Aaaaaand
you
can
also
highlight
songs
or
arBsts
…
and
ﬁlter
for
certain
genres.

http://www.lastloop.de

So,
to
evaluate
the
system
we
followed
the
same
strategy
that
had
already
worked
with

LastHistory.
We
made
the
tool
available
on
the
web
(and
this
Bme
it
was
even
wriXen
in
Java

and
thus
plaporm-‐independent,
while
LastHistory
was
Mac-‐only).
You
could
-‐
and
sBll
can
-‐
run

it
easily
in
your
browser.

For
learning
the
applicaBon,
we
provided
another
ﬁve-‐minute-‐video
that
explained
the
basics
of

interacBon
and
to
capture
the
users’
ﬁndings
we
had
another
short
quesBonnaire
…

…
and
we
also
had
‘feedback’
buXon
in
the
upper
leI
of
the
applicaBon
where
users
could
click

on,
provide
what
they
found
and
send
it
directly
back
to
us.


(3 incomplete)

So,
while
we
were
preXy
convinced
that
we
did
everything
right,
the
response
was
less
than

stellar.
AIer
one
month
we
had
21
responses
to
the
quesBonnaire
and
a
few
with
the
direct

feedback
buXon.

Insights gained:

“That one user is also listening to
a very infamous band from the 70s”

“When did the other user hear my
favorite song, have there been many
connections lately, …”

What
we
found
was
that
people
learned
about
themselves
and
others,
which
was
the
goal
of

the
visualizaBon
and
we
were
happy
that
it
worked.
But
we
wanted
to
ﬁnd
out
what
went

wrong…

Selecting a song was sketchy

Results were cluttered and unclear

…
and
the
problems
were
mostly
due
to
usability
issues
and
the
general
complexity
of
the

applicaBon.
People
found
it
difficult
to
accurately
select
a
song
as
the
selecBon
was
only
based

on
the
horizontal
posiBon
of
the
cursor
and
not
the
verBcal
(so
it
became
very
hard
to
select
a

specific
song
when
zoomed
out).
Also,
people
liked
how
the
results
looked
but
couldn’t
make

much
sense
of
them.
It
was
oIen
just
too
much
informaBon
in
too
liXle
space,
so
drawing
any

insights
other
than
very
superficial
ones
was
difficult.

Screenshot
Of
LastLoop


So
what
we
learned
was
that
even
when
we
ﬁxed
the
usability
issues,
LastLoop
would
probably

sBll
be
more
of
an
expert-‐
than
a
casual
visualizaBon.

Screenshot
Of
LastLoop


Ok,
now
that
you’ve
seen
5
examples
for
visualizaBons
of
listening
histories
that
approached

diﬀerent
aspects
of
the
topic,
where
do
we
go
from
here?

RECOLLECTING
REMINISCING
RETRIEVING
REFLECTING
CACM, May 2010
VisualizaBon
is
nice
and
all,
but
there
is
more
that
we
can
do
with
these
histories.
It’s
nice
to

give
the
creators
of
these
histories
the
chance
to
recollect,
reminisce
and
so
on,
but
we
can
also

use
them
to
make
their
day-‐to-‐day
interacBon
with
music
easier
and
more
convenient.

In
these
last
few
minutes
of
my
talk
I
will
show
you
two
examples
of
how
to
use
this
data
in

other
areas.

One
problem
with
listening
to
music
is
that
there
a
mostly
only
two
ways
to
do
it:
You
either

manually
create
a
playlist
or
pick
an
album
or
have
it
done
fully
automaBcally.
The
former

makes
it
very
tedious
to
listen
to
music
(especially
on
the
go),
while
the
laXer
restricts
you
to

the
choice
of
the
machine
that
might
be
giving
you
the
same
songs
over
and
over
again
and
you

have
very
liXle
inﬂuence
on
that.

Rush
With
our
Rush-‐interacBon
technique
we
wanted
to
create
and
opBon
for
building
playlists

between
the
two
extremes
and
we
called
this
approach
‘repeated
recommendaBons’…

VIDEO

Rush
You
start
just
like
in
the
automaBc
case
with
a
hand-‐picked
seed
song
and
receive
a
set
of
ﬁve

recommendaBons
for
this
item.
Once
you
choose
once
of
these
items,
you
get
another
set
of

ﬁve
and
so
on
and
so
forth.
The
great
thing
about
this
approach
is
that
you
do
not
have
the

large
overhead
of
going
through
your
whole
collecBon
to
create
a
playlist,
but
sBll
have
much

more
freedom
than
in
the
purely
automaBc
case.

So
where
do
listening
histories
come
in
here?
First,
we
can
of
course
use
them
to
shape
the

recommended
items.
In
our
study
we
used
a
pre-‐deﬁned
set
of
music
and
general

recommendaBons
from
last.fm
but
it
would
of
course
make
more
sense
to
adapt
the

recommendaBons
based
on
the
user’s
history….

…
second:
Five
items
is
not
a
lot,
so
it
is
diﬃcult
to
choose
the
right
ones
in
order
not
to

frustrate
the
user.
Having
his
or
her
listening
history
available
means
that
we
can
automaBcally

remove
candidates
that
the
user
does
not
know
(and
would
not
be
very
helpful
in
this

scenario).

RECOLLECTING

Another
thing
that
you
can
do
when
working
with
listening
histories
is
use
them
for
rediscovery

of
music
that
you
forgot.
That
was
something
that
we
oIen
observed
when
people
used
one
of

the
visualizaBons
that
they
were
happy
to
ﬁnd
some
song
or
arBst
that
they
had
forgoXen

about.

But
using
the
visualizaBons
is
an
explicit
acBvity
and
people
commonly
use
diﬀerent
soIware
to

actually
listen
to
music.
So
in
this
last
project,
we
wanted
to
help
them
with
recollecBng
and

reminiscing
while
they
were
actually
listening
to
music.

So
we
decided
to
make
a
plugin
for
a
media
player.
Because
we
wanted
to
keep
it
useful
for
as

many
people
as
possible
we
chose
Songbird,
an
open
source
media
player
with
an
acBve

community,
that’s
available
for
Mac
and
Windows
instead
of
iTunes
or
the
Windows
Media

Player.

Our
idea
for
supporBng
rediscovery
was
based
on
the
idea
that
also
the
Tangle
visualizaBon
was

based
on:
Every
Bme
a
song
appears
in
a
listening
history
it
has
successors
and
predecessors.

And
this
order
of
songs
is
probably
important
for
the
listener,
not
always,
of
course,
but
at
least

someBmes.
So
the
idea
was
to
show
for
the
currently
playing
song
whatever
songs
appeared

before
and
aIer
it.

SongSlope
The
result
looks
like
this:
By
doing
what
they
would
have
done
anyway,
namely
listening
to

music,
users
automaBcally
receive
a
focused
glimpse
into
their
listening
past.
All
songs
before

and
aIer
are
displayed
and
they
can
switch
to
one
of
these
songs
simply
by
clicking
on
them.

SongSlope
…
and
users
can
also
switch
to
a
view
of
the
underlying
listening
sessions,
browse
through
them

or
listen
to
them
as
a
new
playlist.

Currently:

7,200 downloads

(40 partial)

We
had
a
lot
of
downloads
(as
I
said,
Songbird
has
a
very
acBve
community)
but
not
as
many

answers
to
the
quesBonnaire,
probably
because
we
had
no
pop-‐up
or
email
reminder
to
ﬁll
it

out.
We
also
logged
the
relevant
aspects
of
the
user’s
interacBon
with
the
plug-‐in
(of
course,

only
aIer
they
agreed
to
that).

Use cases:

44.8% Re-discovering music

31.0% Generating playlists

We
were
especially
interested
in
what
people
used
it
for
and
found
that
almost
half
of
them

were
able
to
rediscover
music
with
it,
but
also
almost
a
third
used
it
for
creaBng
playlists
(or

relistening
to
old
playlists).
So
even
though
only
a
couple
of
people
answered
the
quesBonnaire

we
got
very
posiBve
feedback
from
them.

Ok,
so
where
does
that
leave
us
and
what
can
you
take
away
from
this
talk:

Listening
histories
are
today
mostly
used
for
recommendaBon.
But
as
they
are
a
type
of

personal
data
that
can
be
easily
collected
and
sBll
can
have
a
powerful
impact
into
people’s

lives
using
them
for
recommending
music
only
is
–
I
think
–
somewhat
of
a
waste.
We
can
do

much
more
with
them.

Screenshot
Of
LastLoop


…
as
you’ve
seen:
We
can
visualize
this
informaBon
to
allow
people
to
reminisce
about
their

past
and
recollect
their
memories,
in
varying
degrees
of
complexity
and
for
various
approaches

to
the
topic…

And
beyond
navel-‐gazing
we
can
also
use
this
data
for
helping
people
with
listening
to
music:

We
can
use
listening
histories
to
improve
the
usability
on
mobile
devcies
for
quickly
and

conveniently
creaBng
personalized
playlists
on
the
go
or
to
add
value
by
lehng
people

painlessly
rediscover
music
while
listening
to
it
anyway.

Genre
……

Sub-Genre

Artists

Albums

Songs

Tags
So,
for
three
more
concrete
results
that
I
learned
while
working
this
topic:
It’s
probably
a
good

idea
to
use
a
Bmeline
as
the
central
metaphor
for
represenBng
personal
histories,
as
the

temporal
aspect
is
very
important
for
ﬁling
this
data
into
one’s
personal
life
story.
Also,

abstracBons
such
as
genre
hierarchies
are
great
for
reducing
the
complexity
of
the
data
while

preserving
the
access
to
single
items.

131

Second,
for
collecBng
results
from
casual
users
several
approaches
can
be
helpful:
We
had

quesBonnaires
that
popped
up
aIer
a
while
in
LastHistory,
we
tracked
relevant
interacBon
with

the
user’s
consent
to
learn
about
how
an
applicaBon
is
used
and
where
it
fails
(in
SongSlope)

and
ﬁnally,
the
feedback-‐buXon
that
we
had
in
LastLoop
allowed
for
impromptu
feedback
with

minimal
overhead.

Finally,
one
very
interesBng
data
source
that
we
tapped
when
creaBng
LastHistory
were
the

user’s
memories.
These
memories
can
give
context
and
meaning
to
plain
lists
of
songs
and
by

using
suitable
memory
triggers
it’s
possible
to
unearth
great
stories
and
understand
these

histories.
Depending
on
the
use
case,
visualizaBon
shouldn’t
underesBmate
the
value
of
having

a
real
person
sihng
in
front
of
the
machine.

I
think
the
central
part
is
that
these
histories
are
reflecBons
of
their
creators’
lives:
Music

accompanies
them
during
their
good
and
their
bad
Bmes,
their
triumphs
and
their
tragedies
and

forms
an
inseparable
bond
with
these
events.
But
what
they
are
lacking
are
the
tools
to
use

them
in
the
same
way
that
they
use
photos
for
reflecBng
about
their
past
and
making
sense
of

their
lives.
So
I
hope
my
work
is
a
first
step
towards
giving
this
data
back
to
the
people
who

created
it.

DOMINIKUS BAUR
UNIVERSITY OF dominikus.baur@iﬁ.lmu.de
MUNICH (LMU), twitter: @dominikus
GERMANY

Thank
you!

The Songs of Our Past

Recomendados

Recomendados

Más contenido relacionado

Último

Último (20)

The Songs of Our Past