Slides from my talk at UCD2012 (London) and UX Cambridge 2012.
Case study of how I run research at music start up Songkick and insight into our product development process
Link to a video of the same talk at Bunnytalk
http://www.bunnyfoot.com/blog/?p=1886&preview=true (15 mins - excluding Q&A)
2. • Live music start up
• 2nd biggest live music site after Ticketmaster
• Fans track their favourite artists so they never
miss them live
• 7 million fans discover concerts on Songkick
every month
3. The usual things
• Daily stand ups
• Kanban board
• Short iteration cycles
• Prototypes as the guide for what we are building.
• Sketching hacks with designers, PM’s and devs
• Post it notes and writeboards everywhere.
• Define and measure the success of everything
4. Product development inspiration
• Flow Interactive - UCD
• Jeff Gothelf and Josh Seiden - Lean UX
• Marty Cagan - Silicon Product Group
• Tom Chi - Experience Lead at Google X
• Jeff Paton - Dual track Scrum
• Jake Knapp - Google Ventures
• Eric Reis and Steve Blank - Lean start up
5. Customer discovery Product discovery Product delivery
Listen Experiment Execute
How we make stuff at Songkick
6. Steve Blank - Customer Discover
• Is this worth working on?
• Is there anyone out there who wants this?
• If we could build a perfect solution to this
problem, would anyone buy* it?
Steve Blank circa 1978
7. Input
Customer discovery Output
Idea for a product Customers behaviours,
area of product feature attitudes, goals, motivations,
frustrations and pain points
Identify Some validation that this is
customers worth working on and there
are people out there who
want this.
Meet with
them
Communicate what
you learnt to the
team
Synthesis all the
data / stories /
artefacts you
collected
Is this worth working on?
Is there anyone out there who wants this?
8. Input
Product discovery Output
Product / Biz Goal Validated learnings go
Customer problem into the product delivery
backlog
Ideas /
concepts
Hypothesis
Analyse results
Design
experiments
Run experiments
How can we make it work?
What solution(s) can we provide?
Will people be able to use them?
9. Product discovery: hypothesis
If we show bands a share on Facebook button after they have
added a tour date
then bands will share their events to Facebook,
because they want to tell their fans about their shows.
10. Input
Product discovery Output
Product / Biz Goal Validated learnings go into
Customer problem the product delivery
backlog
Ideas /
concepts
Hypothesis
Analyse results
Design
experiments
Run experiments
11. Product discovery: experiments
• Build the solution
• In-product prototype
• Live data prototype (outside product)
• Clickable prototype
• Flat mock up
• Paper prototype
• + lots of other creative experiments - no code
or interface needed
12. Input
Product discovery Output
Product / Biz Goal Validated learnings go into
Customer problem the product delivery
backlog
Ideas /
concepts
Hypothesis
Analyse results
Design
experiments
Run experiments
13. Inputs Product delivery
Validated product
discovery ideas
1000’s of users Ideas
requesting the same
feature
Define
success criteria
Competitor product Prototype (qual/quant/KPI)
features
It’s still a iterative
Refine Build process
etc.
Test
Prioritise
improvements
Ship
Measure
How can we build this efficiently?
How can we ensure what we are
building is of good enough quality?
14. Facebook share experiment
If we show bands a share on Facebook button after they have
added a tour date
then bands will share their events to Facebook,
because they want to tell their fans about their shows.
15. Customer discovery Product discovery Product delivery
Listen Experiment Execute
How can we make it work?
Is this worth working on? How can we build this efficiently?
What solution(s) can we provide?
Is there anyone out there How can we ensure what we are
Will people be able to use our
who wants this? building is of good enough quality?
solutions to solve the problem?
30% 50% 20%
Right now as a UXD at Songkick
this is how I spend my time
22. Meeting room Free - mic boom and screwdriver for over
the shoulder mobile testing
Free Window frames we
found in the basement -
lying around from office
refurb projects
£300 - knock a hole
in the wall, fit the frame
and ordinary glass
£30 - Car window tint film
(used by boy racers but
also makes a pretty good
budget two way mirror)
Free - mic I stole from the
unused conference calling
set up.
23. Audio and video cables - Maplin £60
Free - Old mini-hifi - donated by a
Songkick developer and used as a
pre amp to increase the volume of
the sound from meeting room.
4 sets of headphones - £15 each
but gradually upgraded to better
sets (donated by people who work
at Songkick)
4 way audio splitter £50 Maplin
Observation room
27. We run 3 - 6 users every Wednesday
12
People
Hello! 0 people gets you 0 insights
0
0 Insights alot
28. 3 websites, 2 mobile apps and Spotify app
Fan apps Data and Crowd sourced
industry touring
Songkick.com
iPhone app
Android app detour-ldn.songkick.com
Spotify app tourbox.songkick.com detour.songkick.com
29. 3 websites, 2 mobile apps and Spotify app
Fan apps Data and industry Crowd sourced touring
Design team
QA
Support team
30. Train the product managers to run
research sessions
• Product managers have natural empathy for
customers, are great communicators and enjoy
talking to people
• The more people that can run sessions the more
feedback you’ll get
32. Agree what are the most important things
to learn and uncover
• Write them down so you can refer to them later.
• Structure sessions so 1/3 to 1/2 are around
customer discovery
• Rest of session is for feedback on prototypes and
evaluating things we recently shipped.
34. Few example of the ones we use regularly
Self-Moderated Survey tools Click and memory Analytics
remote tools test tools
Google Analytics and log file analysis
35. Remote tools are just another data point
• Sit along side our qualitative research
• Help us build a clearer picture of our customers
37. Recruiting is hard
• We all know that the research we run is only as
good as the people who we choose to participate
• Getting the recruit right is important!
• We always write a recruitment screener
38. Recruiting yourself - you know your
customers better than anyone
• Existing customers - mine support desk, Twitter
and Facebook
• Target customers - place ads where they already
hang out
• Post ads on Gumtree
• Go to concert venues
• Get people who work in music shops to help
• Post on musician sites / Contact music colleges
• Identify participants who can help with future
recruits
39. The 2 for 1 recruiting trick
Place ad on Survey on
Gumtree Polldaddy
5 participants for
your next study
Include specific Include
date, outline how
the session
recruitment
screener
=
works and offer questions and 200 responses
a small incentive larger survey for your survey.
questions
42. How we get feedback to the team fast
• Allow 30 mins between each session
• Get observers to help out with the analysis
• Tweak prototypes or the live site between
sessions to learn as much as possible
• Write top 5's after each session and share them
with the team right away
43. A little on how I do analysis
• Record audio of sessions with Pearnote
• Record all the sessions with person in picture,
audio and what happened on screen but rarely get
a chance to watch them :(
• Want to figure out a way to make highlights video
fast - WIP.
• Report back on research goals + hypothesis in 24
hrs.
• Sometimes afterwards I build customer
storyboards to use in persona work later.
46. Minimum viable research
• Even with a small team, a little cash and limited
time you can find a way to run research every
week.
1. Get out the building
2. Stay in the building and hack your space
3. Get feedback from your target audience as often as you can
4. Always know what you're trying to learn before you start
5. Experiment with remote research tools to fill in the gaps
6. Hustle the recruit
7. Get feedback to the team as soon as possible
8. Just do it
47. By making research easy to do
regularly and turning around
analysis fast, user research is
now a key part of how we make
product and strategic business
decisions.
48. Your responsibility as people designing
products and services is to step out of
your own perspective and immerse
yourself in the world of the people you
are designing for.
50. Thanks
Jo Packer
@jujupacker
jo@songkick.com
jujupacker@googlemail.com
51. Links to people who have
inspired my approach to making
products
Flow Interactive - UCD
LUXR - Lean UX
Jeff Gothelf and Josh Seiden - Lean UX
Marty Cagan - Silicon Valley Product Group
Tom Chi - Experience Lead at Google X
Jeff Paton - Dual track Scrum
Jake Knapp - Design Staff
Eric Reis - Lean start up
Steve Blank - Lean start up