Music recommendation is broken - automatic music recommenders make mistakes that no human would ever make. In this talk, we will explore why recommenders make such dumb mistakes and we will explore some of the new ideas coming from recommendation and music researchers to help make music recommendations better. Slides from the SXSW 2009 Panel. Speakers: Paul Lamere from The Echo Nest, and Anthony Volodkin from The Hype Machine.

1. Help! My iPod
thinks I’m emo.
SXSW Interactive
March 17, 2009
#sxswemo
Paul Lamere
Anthony Volodkin Photo (CC) by Jason Rogers
Monday, March 23, 2009

2. Music recommendation is broken
A recommendation that no human would make
If you like Britney Spears ...
You might like the Report on
Pre-War Intelligence
Monday, March 23, 2009

3. Help! My iPod thinks I’m emo
What we are talking about today
Important?
Broken?
Can we fix it?
Q/A
Monday, March 23, 2009

4. Why do we care?
Compulsory Long Tail slide
Monday, March 23, 2009

5. Why do we care?
Compulsory Long Tail slide
Monday, March 23, 2009

6. Why do we care?
Compulsory Long Tail slide
Monday, March 23, 2009

7. Why do we care?
Compulsory Long Tail slide
Monday, March 23, 2009

8. State of music discovery
We can’t seem to find the long tail
Sales data for 2007
- 4 million unique tracks sold
But ...
- 1% of tracks account for 80% of sales
- 13% of sales are from American Idol or Disney artists
State of the Industry 2007 - Nielsen Soundscan
Monday, March 23, 2009

9. State of music discovery
We can’t seem to find the long tail
Sales data for 2007
- 4 million unique tracks sold
But ...
- 1% of tracks account for 80% of sales
- 13% of sales are from American Idol or Disney artists
State of the Industry 2007 - Nielsen Soundscan
Monday, March 23, 2009

10. State of music discovery
We can’t seem to find the long tail
Sales data for 2007
- 4 million unique tracks sold
But ...
- 1% of tracks account for 80% of sales
- 13% of sales are from American Idol or Disney artists
State of the Industry 2007 - Nielsen Soundscan
Make everything available
Monday, March 23, 2009

11. State of music discovery
We can’t seem to find the long tail
Sales data for 2007
- 4 million unique tracks sold
But ...
- 1% of tracks account for 80% of sales
- 13% of sales are from American Idol or Disney artists
State of the Industry 2007 - Nielsen Soundscan
Make everything available
Help me find it
Monday, March 23, 2009

12. Help! I’m stuck in the head
The limited reach of music recommendation
Popularity
83 Artists 6,659 Artists 239,798 Artists
Sales Rank
Study by Dr. Oscar Celma - MTG UPF
Monday, March 23, 2009

13. Help! I’m stuck in the head
The limited reach of music recommendation
48% of recommendations
Popularity
83 Artists 6,659 Artists 239,798 Artists
Sales Rank
Study by Dr. Oscar Celma - MTG UPF
Monday, March 23, 2009

14. Help! I’m stuck in the head
The limited reach of music recommendation
48% of recommendations
Popularity
52% of recommendations
83 Artists 6,659 Artists 239,798 Artists
Sales Rank
Study by Dr. Oscar Celma - MTG UPF
Monday, March 23, 2009

15. Help! I’m stuck in the head
The limited reach of music recommendation
48% of recommendations
Popularity
0% of
recommendations
52% of recommendations
83 Artists 6,659 Artists 239,798 Artists
Sales Rank
Study by Dr. Oscar Celma - MTG UPF
Monday, March 23, 2009

16. Help! My iPod thinks I’m emo
Why is music recommendation broken?
Monday, March 23, 2009

17. The Wisdom of Crowds
How does collaborative filtering work?
35% 4% 62% 8% 60%
Overlap Data based on listening behavior of 12,000 Last.fm Listeners
Monday, March 23, 2009

18. The stupidity of solitude
The Cold Start problem
If you like Blondie, you might like the DeBretts ...
But the recommender will never tell you that.
Monday, March 23, 2009

19. The stupidity of solitude
The Cold Start problem
If you like Blondie, you might like the DeBretts ...
But the recommender will never tell you that.
Monday, March 23, 2009

20. The stupidity of solitude
The Cold Start problem
If you like Blondie, you might like the DeBretts ...
0%
0%
0%
But the recommender will never tell you that.
Monday, March 23, 2009

21. The Harry Potter Problem
If you like X you might like Harry Potter
Monday, March 23, 2009

22. The Harry Potter Problem
If you like X you might like Harry Potter
Monday, March 23, 2009

23. Popularity Bias
Rich get richer - diversity is the biggest loser
Results of popularity bias:
- Rich get richer
- Loss of diversity
- No long tail recommendations
Monday, March 23, 2009

24. Popularity Bias
Rich get richer - diversity is the biggest loser
Results of popularity bias:
- Rich get richer
- Loss of diversity
- No long tail recommendations
Monday, March 23, 2009

25. The Novelty Problem
If you like The Beatles you might like ...
Monday, March 23, 2009

26. The Napoleon Dynamite Problem
Some items are not easy to categorize
Monday, March 23, 2009

27. The Opacity Problem
“If you like NiN you might like Johnny Cash” - why?
Why???
Monday, March 23, 2009

28. Hacking the recommender
Recommenders can be easily manipulated for fun or profit
Monday, March 23, 2009

29. Hacking the recommender
Recommenders can be easily manipulated for fun or profit
Monday, March 23, 2009

30. Help! My iPod thinks I’m emo
Fixing music recommendation
Monday, March 23, 2009

31. Fixing music recommendation
Eliminating popularity bias and feedback loops
Tim Eric Jim Brian
Adam Paul
Aaron Peter Chris Liz Yury Todd
Bethe Kirk Erik Tristan Sue Jen Todd
Jason
Scotty Erik
Cari Jason
Monday, March 23, 2009

32. Fixing music recommendation
Semantic-based recommendation
pop legend dance diva sexy
american guilty pleasure
teen pop 00s soul
90s pop rnb
dance-pop
rock rock dance-pop soul
singer-songwriter
hot 90s
emo alternative
Monday, March 23, 2009

33. Fixing music recommendation
Where does this information come from?
Playlists
Artist Bios
The web
Reviews
Blogs
Lyrics
pop legend dance diva Forums
sexy american guilty Events
pleasure 90s teen pop 00s
rnb pop rock rock singer-
songwriter dance-pop soul emo Social sites
hot 90s
alternative
female fronted metal dark
rock alternative goth metal metal
goth rock emo gothic dark
heavy metal gothic
gothic rock
metal hard rock melodic metal
Crawler
symphonic metal rock metal
pop nu
Monday, March 23, 2009

34. Fixing music recommendation
Content-based recommendation
4
x 10
2
1
0
! -1
! -2
0 5 10 15 20 25 sec.
BPM
190
143
114
96
72
60
0 5 10 15 20 25 sec.
1
0.8
0.6
0.4
0.2
0
60 72 96 114 143 BPM 190 240
Monday, March 23, 2009

35. Content-based recommendation
Using machines to listen to music
Perceptual features audio: Pattern layer
- time signature / tempo linear ratio
square ratio
- key/ mode Beat layer
1:4
- timbre 1:16
- pitch Segment layer ~1:3.5
- loudness ~1:12.25
- structure ~1:40
Frame layer ~1:1600
1:512
1:262144
Audio layer
4
x 10
2
1
wave form
0
! -1
! -2
0 0.5 1 1.5 2 2.5 3 sec.
25
25-band spectrogram
20
15
10
5
1
0 0.5 1 1.5 2 2.5 3 sec.
1
0.8
Monday, March 23, 2009
ss
0.6

36. Hybrid Recommendation
The best of all worlds
listener data Hybrid Recommender
preference editorial
user history
35% 4% 62% 8% 60%
recommendations
12% 18% 17% 34% 21%
audio data 4% 49% 5% 48% 7%
artist
collaborative filterer
audio
analysis
track
web data
content-based
user
adj Term K-L bits np Term K-L bits
random guessing is:
aggressive 0.0034 reverb 0.0064
softer 0.0030 the noise 0.0051
synthetic 0.0029 new wave 0.0039
a Na
KL = log punk 0.0024 elvis costello 0.0036
(a + b) (a + c)
N sleepy 0.0022 the mud 0.0032
cultural
funky 0.0020 his guitar 0.0029
b Nb
+ log noisy 0.0020 guitar bass and drums 0.0027
(a + b) (b + d)
N angular 0.0016 instrumentals 0.0021
analysis
c Nc acoustic 0.0015 melancholy 0.0020
+ log romantic 0.0014 three chords 0.0019
(a + c) (c + d)
N
d Nd Table 2. Selected top-performing models of adjective and
+ log
(b + d) (c + d)
semantic-based
N noun phrase terms used to predict new reviews of music
with their corresponding bits of information from the K-L
(3)
distance measure.
This measures the distance of the classifier away from a
degenerate distribution; we note that it is also the mu-
7.4. Review Regularization
tual information (in bits, if the logs are taken in base 2)
Monday, March 23, 2009

37. What if?
What if technology isn’t the answer?
No magic recommender?
photo by mebajason (cc)
Monday, March 23, 2009

38. Taste is irrational
What makes people like things?*
Social connections:
Who else likes this? Are they _____?
How many of them are there?
Context:
Purpose / Surroundings
Medium, related info, meaning
* Most won’t admit this if you ask
Monday, March 23, 2009

39. Not all listeners equal
Different types of listeners need different kinds of recommenders
Monday, March 23, 2009

40. Not all listeners equal
Different types of listeners need different kinds of recommenders
Indifferents 40%
Monday, March 23, 2009

41. Not all listeners equal
Different types of listeners need different kinds of recommenders
Casuals 32%
Indifferents 40%
Monday, March 23, 2009

42. Not all listeners equal
Different types of listeners need different kinds of recommenders
Enthusiasts 21%
Casuals 32%
Indifferents 40%
Monday, March 23, 2009

43. Not all listeners equal
Different types of listeners need different kinds of recommenders
Savants 7 %
Enthusiasts 21%
Casuals 32%
Indifferents 40%
Monday, March 23, 2009

44. Accept irrationality
To make real discoveries possible:
Offer unique presentation
Show context
Create meaning
Monday, March 23, 2009

45. Unique Presentation & Editorial
Ishkurs’ guide to Electronic Music (http://techno.org)
Monday, March 23, 2009

46. Context & Interaction
Guitar hero / Rock Band gameplay
Monday, March 23, 2009

47. Context
What’s on her playlist? imeem (http://imeem.com), social playlists
Monday, March 23, 2009

48. Context & Meaning via Editorial
14 Tracks: Weekly themed selections (http://14tracks.com)
14 Tracks that make you wish you played the piano:
Monday, March 23, 2009

49. Playful Presentation & Meaning
thesixtyone (http://thesixtyone.com)
Monday, March 23, 2009

50. Visible Context & Meaning
The Hype Machine
Monday, March 23, 2009

51. Meaning via Authoritative Editorial
Pitchfork’s Best New Music pages (http://pitchforkmedia.com)
Monday, March 23, 2009

52. The Future of Music Discovery
Listen, Learn, Understand, Engage - for all music.
Monday, March 23, 2009

53. The Future of Music Discovery
Listen, Learn, Understand, Engage - for all music.
Monday, March 23, 2009

54. The Future of Music Discovery
Listen, Learn, Understand, Engage - for all music.
Monday, March 23, 2009

55. The Future of Music Discovery
Listen, Learn, Understand, Engage - for all music.
Monday, March 23, 2009

56. The Future of Music Discovery
The challenge
Music Music
I Like You Like
Music
I Used
To Like
Get this t-shirt at dieselsweeties.com
Monday, March 23, 2009

57. The Future of Music Discovery
Editors, people, serendipity. Scaled.
image by luc legay (CC)
Monday, March 23, 2009

58. Help! My iPod
thinks I’m emo.
Questions?
Paul Lamere
the.echonest.com
Paul@echonest.com
MusicMachinery.com
Anthony Volodkin
hypem.com
anthony@hypem.com
fascinated.fm
Photo (CC) by Jason Rogers
Monday, March 23, 2009