Early stage-web-product-management-by-dan-olsen-090728040916-phpapp02

Copyright © 2009 YourVersion
Dan OlsenDan Olsen
CEO, YourVersionCEO, YourVersion
July 24, 2009July 24, 2009
Early Stage Web Product Management

What You Wanted to Hear About
1 2 3 4 5 6 7 8 9 10
Usability
Understanding user
needs
Prioritizing features
Translating learnings
from metrics into
action
Maximizing ROI on
development
resources
Setting up and
tracking the right
metrics
UI design

AgendaAgenda
Understanding customer needs
Prioritizing, scoping, and maximizing ROI
on engineering resources
UI design
Usability testing
Using metrics for optimization
Will post slides to slideshare.net/dan_o

My Background
Education
BS, Electrical Engineering, Northwestern
MS, Industrial Engineering, Virginia Tech
MBA, Stanford
PHP, MySQL, JavaScript, XHTML, CSS, UI design
18 years of Product Management Experience
Managed submarine design for 5 years
5 years at Intuit, led Quicken Product Management
Led Product Management at Friendster
Olsen Solutions LLC, PM consultant for startups
CEO & Founder of YourVersion, real‐time discovery startup

For an Early Stage Startup,
Which is More Important?
Qualitative Qualitative
Information?Information?
Quantitative Quantitative
Information?Information?
oror
“When you’re Small,
start with Qual!”

How is developing Early Stage Products
Different from Later Stage Products?
With an early stage product, you have
WAY MORE uncertainty about:
Who your target customers really are
Which customer needs you should address
How to best meet those needs
What design works best
These are qualitative learnings/decisions
Quantitative is also valuable (later)

What is Product Management?

Product Management is
Critical Link in Value Creation
Market
• Current
customers
• Prospective
customers
• Competitors
Product
Management
Development
Team

The Product Manager’s Job:
A Successful Product
Be the expert on the market and the customer
Translate business objectives and customer
needs into product requirements
Be the clearinghouse for all product ideas
Work with team to design & build great product
Define and track key metrics
Identify, plan & prioritize product ideas to
maximize ROI on engineering resources

What’s the Formula
for a Winning Product?
A product that:
Meets customers’ needs
Is better than other alternatives
Is easy to use
Has a good value/price

Russians: pencil
NASA: space pen
($1 M R&D cost)
Example:
Ability to write in space
(zero gravity)
Problem Space vs. Solution Space
Problem Space
A customer problem,
need, or benefit that the
product should address
A product requirement
Solution Space
A specific
implementation to
address the need or
product requirement

Product Level
Problem Space
(user benefit)
Solution Space
(product)
TurboTax
TaxCut
Pen and
paperPrepare
my taxes
File my
taxes

Feature Level
Problem Space
(user benefit)
Solution Space
(feature)
Gmail
importerMake it easy
to share a
link with my
friends
Allow me to
reuse my
email
contacts
Design
#1
Design
#2
Design
#3
Design
Preview with
checkboxes
User can edit
before import
#1 No No
#2 Yes No
#3 Yes Yes

Types of Customer Benefits
Functional Benefits
Allow the user to do something they couldn’t do before
Deliver a benefit more quickly, conveniently, or cheaply
Booking travel more easily (travel sites)
Talking for less money (VOIP software)
Emotional Benefits
Control (Quicken)
Feeling informed (news sites)
Confidence / security (anti‐virus software)
Enjoyment / entertainment (YouTube)
Self‐expression (MySpace, Facebook)

The Customer Benefits “Ladder”
HigherHigher‐‐levellevel
benefitbenefit
(more abstract)(more abstract)
Quicken makes it easy for me Quicken makes it easy for me
to balance my checkbookto balance my checkbook
……which gives me a clear picture which gives me a clear picture
of how much money I haveof how much money I have
……which makes me feel more in which makes me feel more in
control of my financescontrol of my finances
……which means one less thing to which means one less thing to
worry about in my hectic lifeworry about in my hectic life
LowerLower‐‐levellevel
benefitbenefit
(more specific)(more specific)

How Do You Prioritize User Benefits
and Product Features?
Need a framework for prioritization
Which user benefits should you address?
Which product features to build (or improve)?
Importance vs. Satisfaction
Importance of user need (problem space)
Satisfaction with how well a product meets the
user’s need (solution space)
Opportunity =
High Importance need with low Satisfaction

What is Your Value Proposition?What is Your Value Proposition?
Which user benefits are you going to provide?Which user benefits are you going to provide?
How will your product be better than rest?How will your product be better than rest?
Benefit 1
ImportanceImportance
to Userto User
Current UserCurrent User
SatisfactionSatisfaction
Benefit 2
User BenefitUser Benefit
UpsideUpside
PotentialPotential
Benefit 4
Benefit 5
Benefit 6
High
High
Med
Low
Low
Low High – +
Doesn’t matter
Benefit 3 Low
opportunity
opportunity
?
?

Importance vs. Satisfaction
Ask Users to Rate for Each Feature
98
8784
8679 847055 80
72
80
75
41
50
55
60
65
70
75
80
85
90
95
100
40 50 60 70 80 90 100
Satisfaction
Importance
Recommended reading: “What
Customers Want” by Anthony Ulwick
BadBad
GreatGreat

Kano Model: User Needs & Satisfaction
User SatisfactionUser Satisfaction
User DissatisfactionUser Dissatisfaction
Performance
(more is better)
Delighter (wow)
NeedNeed
not metnot met
NeedNeed
fully metfully met
Must Have
Needs & features
migrate over time

Is the site up when I want to use it?
Is the site fast enough?
Does the functionality work?
Does the functionality
meet my needs?
Olsen’s Hierarchy of Web User Needs
(adapted from Maslow)
Customer’s Perspective What does it mean to us?
Uptime
Page Load Time
Absence of Bugs
Feature Set
Usability & Design
Decreasing
Dissatisfaction
Increasing
Satisfaction
How easy to use is it?

Prioritization and Scope
Customer value is only half the equation
How much engineering effort will it take?
Need to consider value and effort (ROI)
Ruthlessly prioritize: rank order
Be deliberate about scope & keep it small
It’s easy to try to do too much
Strategy = deciding what you’re NOT doing
Break features down into smaller chunks
Smaller scope → faster iterations → better

Have to Prioritize Across Multiple
Dimensions At The Same TimeCustomer ValueCustomer Value
TimeTime
Customer Customer
UnderstandingUnderstanding
Functionality
Quality
Ease of Use

Prioritizing Product Ideas by ROIPrioritizing Product Ideas by ROI
Investment (developer‐weeks)
Return (Value Created)
Idea C
Idea B
Idea D
Idea A
Idea F
1
1
2
3
4
2 3 4
?

User Benefits vs. Ease of Use
Q: If two products equally deliver the exact
same user benefits, which product is better?
A: The product that’s easier to use
‘Ease of use’ provides benefits
Saves time
Reduces cognitive load
Reduces frustration
Not many companies excel at UI design
‘Ease of use’ can be differentiator

The Design Gap at Most Start‐ups
Define Design Code
Product Mgmt Engineering
PM Eng
EngPM
UI
Level
1
2
3
4

The UI Design IcebergThe UI Design Iceberg
Visual
Design
Interaction
Design
Information
Architecture
Conceptual
Design
Recommended reading: Jesse James Garrett’s
“Elements of User Experience” chart, free at www.jjg.net
What most
people see
and react to
What good
PMs and
Designers
think about

Elements of User Interface Design
Consists of Three Distinct Elements:
Information Architecture
Structure and layout at both site and page level
How site is structured (sitemap)
How site information is organized (site layout)
How each page is organized (page layout)
Interaction Design
How user and product interact with one another
User flows (e.g., navigation across multiple pages)
User input (e.g., controls and form design)
Visual Design
“How it looks” vs. “What it is”, often called “chrome”
Fonts, colors, graphical elements

Documents used
Sitemap
Show how sections of website are organized
Show major navigation patterns
Wireframes
Show the layout of components on a page
Does NOT focus on visual design
Black & White
No graphics
Templates for overall website and individual pages
Tools: Visio, OmniGraffle, Axure, Powerpoint, Word,
Excel, Photoshop, Balsamiq, WriteMaps, whiteboard

Sitemap

Wireframe

Napkin Wireframe

Interaction Design
Documents used
Flowchart
Combination of Wireframes & Flowcharts
Tools: Visio, OmniGraffle, Powerpoint,
Photoshop, whiteboard
May build prototype using HTML, jQuery,
Ruby on Rails, Flash, or paper
Usability testing can help find problems

Flowchart showing conditional logic

Napkin Flowchart

Combo Wireframe & Flowchart

Visual Design
Documents used
Mockups (aka comps)
Tools used: Photoshop, Illustrator,
Fireworks
Designer may also deliver HTML & CSS
version (with no back‐end code)

Visual Design
Mockup

Visual Design
UI Spec

Visual Design
Specifying Color Palette

The Value of Usability Testing
Critical to talk with customers 1‐on‐1
Gain better understanding of
Usability issues with your product
Customer needs and problems
What alternatives customers are using,
pros & cons of each, customer preferences
QA: use cases & bugs you haven’t seen
Really a “user learning” session
Make test as real for user as possible

“Ramen” Usability for Startups
Anyone can do it!
Ingredients:
1 user with their laptop
1 desk
1 person to conduct usability
Pen and paper
N optional observers
N+2 chairs

Typical Format for Customer Session
5 ‐ 10 min: Ask questions to understand user
needs and solutions they currently use
30 ‐ 50 min: Usability test
Non‐directed as much as possible
When necessary, direct user to attempt to
perform a specific task
5 ‐ 10 min: Wrap‐up
Answer any user questions that came up
Point out/explain features you want to highlight

Dos & Don’ts of Conducting Usability
Do
Explain to the user:
Their usability test will help improve the product
Not to worry about hurting your feelings
“Think Aloud Protocol”
Ask user to attempt the task, then be a fly on the wall
Ask non‐leading, open‐ended questions
Take notes and review them afterwards for take‐aways
Don’t
Ask leading questions
“Help” the user or explain the UI (e.g., “click over here”)
Respond to user frustration or questions (until test is over)
Get defensive
Blame the user

Usability Case Study: Travel Sites
Consider 3 major travel sites
Expedia, Travelocity, Orbitz
All 3 try to provide same user benefits
Easily find flights that match your schedule
Easily find the lowest price flights
UI Design differences make some better
Case study: round trip from San Fran to NYC
Want to find best combo of price and airports
3 possible airports for each city
9 possible one‐way airport combinations
81 possible round‐trip airport combinations

Expedia:
Only One Airport Combo at a Time
•• Have to Have to
manually manually
check all 9 check all 9
comboscombos
••3 clicks to 3 clicks to
change airportchange airport
••Then wait for Then wait for
new resultsnew results
••24 clicks +24 clicks +
8 page reloads 8 page reloads
to see all 9 to see all 9
comboscombos

Travelocity:
Can Select which Airports to Include

Travelocity:
Can only view results 1 combo at a time
•• Clicking button brings up Clicking button brings up
results for this comboresults for this combo
••Problem: browser Back Problem: browser Back
button loses other button loses other
airports!airports!
••Have to go through Have to go through
‘‘Change SearchChange Search’’ process =process =
9 clicks + 4 page reloads 9 clicks + 4 page reloads
for each combofor each combo
••72 clicks + 32 page loads 72 clicks + 32 page loads
to see other 8 combosto see other 8 combos

Orbitz:
Binary choices to include nearby airports
•• Good news: all combos Good news: all combos
appear in resultsappear in results
••8080‐‐mile radius might mile radius might
include airports I doninclude airports I don’’t t
want (3 for SFO, 5 for NYC) want (3 for SFO, 5 for NYC)
but turned out not to be but turned out not to be
an issuean issue

All 3 Travel Sites have Identical
“Airline vs. Number of Stops” Price Grids
TravelocityTravelocity
OrbitzOrbitz
ExpediaExpedia

Options for Sorting Flight Results
Website
Can Sort By Expedia Travelocity Orbitz
Airline Y
Departure Time Y Y Y
Arrival Time Y Y
Travel Time Y Y Y
Price Y Y Y

Summary Comparison of Travel Sites
User Benefit Expedia Travelocity Orbitz
Ability to include other
nearby airports
Yes Yes Yes
Flight Results Sorting
Options
Med High Low
Ability to pick specific
nearby airports
High
(by changing)
High
(can pre‐select)
Low
Ease of trading off
airport combos vs. price
Low High Med
Ease of seeing results
for multiple airports
Med Low High
Airline vs. Number of
Connections Price Grid
Yes Yes Yes
Overall ability to easily
find best airport combo
Med Low High

And now,And now,
for something completely for something completely
differentdifferent……
Quantitative!Quantitative!
(finally)(finally)

Approaching Business as an Approaching Business as an
Optimization ExerciseOptimization Exercise
Given reality as it exists today,Given reality as it exists today,
optimize our business resultsoptimize our business results
subject to our resource constraints.subject to our resource constraints.

Profit = Profit = RevenueRevenue ‐‐ CostCost
Unique VisitorsUnique Visitors x  x  Ad Revenue per VisitorAd Revenue per Visitor
Impressions/VisitorImpressions/Visitor x  Effective CPM / 1000x  Effective CPM / 1000
Visits/Visitor  x  Visits/Visitor  x  PageviewsPageviews/Visit  x  Impressions/PV/Visit  x  Impressions/PV
New VisitorsNew Visitors + Returning Visitors+ Returning Visitors
Invited VisitorsInvited Visitors + Uninvited Visitors+ Uninvited Visitors
# of Users Sending Invites  x  Invites Sent/User  x  Invite Conv# of Users Sending Invites  x  Invites Sent/User  x  Invite Conversion Rateersion Rate
Define the Equation of your BusinessDefine the Equation of your Business
““Peeling the OnionPeeling the Onion””
Advertising Business Model:

( SEO Visitors + SEM Visitors + Viral Visitors )  x  Trial Conve( SEO Visitors + SEM Visitors + Viral Visitors )  x  Trial Conversion Ratersion Rate
Paying UsersPaying Users x  x  Revenue per Paying UserRevenue per Paying User
New Paying UsersNew Paying Users +  +  Repeat Paying UsersRepeat Paying Users
Previous Paying Users  x  ( 1 Previous Paying Users  x  ( 1 –– Cancellation Rate )Cancellation Rate )Trial UsersTrial Users x  x  ConvConv RateRate
Profit = Profit = RevenueRevenue ‐‐ CostCost
Equation of your BusinessEquation of your Business
Subscription Business ModelSubscription Business Model

How to Track Your Metrics
Track each metric as daily time series
Create ratios from primary metrics: X / Y
Example: How good is your registration page?
Okay: # of registered users per day
Better: registration conversion rate =
# registered users / # uniques to reg page
Date
Unique
Visitors
Page
views
Ad
Revenue
New User
Sign‐ups
…
4/24/08 10,100 29,600 25 490
4/25/08 10,500 27,100 24 480
…

Sample Signup Page Yield DataSample Signup Page Yield Data
Daily Signup Page Yield vs. Time
New Registered Users divided by Unique Visitors to Signup Page
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
1/31 2/14 2/28 3/14 3/28 4/11 4/25 5/9 5/23 6/6 6/20 7/4 7/18 8/1 8/15 8/29 9/12 9/26 10/1
0
DailySignupPageYield
Changed
messaging
Added questions
to signup page
Started requiring
registration

Identifying the Identifying the ““Critical FewCritical Few”” MetricsMetrics
What are the metrics for your business?
Where is current value for each metric?
How many resources to “move” each metric?
Developer‐hours, time, money
Which metrics have highest ROI opportunities?
ReturnReturn
InvestmentInvestment
ReturnReturn
ReturnReturn
Metric AMetric A
Good ROIGood ROI
Metric BMetric B
Bad ROIBad ROI
Metric CMetric C
Great ROIGreat ROI

Using Web Analytics Tools to
Understand Your Users
Tracking visitors and traffic
Seeing where users are clicking
Measuring key conversions
Monitoring user feedback

Basic Tracking of Traffic:
Google Analytics
•Unique
visitors
•New vs.
returning
•Pageviews
•Time on site
•Top referrers
•Top geos

Seeing Where Users are Clicking:
CrazyEgg Heatmap
•Shows Click
Density
•Color indicates
% of clicks
•See which links
perform best
•See impact of
UI changes

Measuring Key Conversions:
Conversion Funnel
•Tie user actions to
business goals
•Instrument key steps in
user flow
•See where users are
dropping off
•Quantify improvement
from changes

Monitoring User Feedback
Quantitatively with Kampyle
Simple popup
Unobtrusive
Solicitation Average Grade over time
Average Grade
for each page
on your site

UI questions are never yes/no! (not binary)
Should ask: “What percentage of users …?”
UI changes impact your metrics
Impact can be positive, negative, small, large
Seek high‐ROI UI changes
Typical UI design question:
“When using web pages, do users scroll down?”
‐ Yes
‐ No
Approaching UI Design Analytically

The Fold: “Chrome” Takes Away Pixels
768
pixels
1024 pixels
Only
600
pixels

Copyright © 2009 YourVersionCopyright Copyright ©© 2009 Olsen Solutions LLC2009 Olsen Solutions LLC
Put Key Conversion Actions Above The FoldPut Key Conversion Actions Above The Fold
Landing Page ALanding Page A Landing Page BLanding Page B
Key conversion action
is above the fold
Key conversion action
is below the fold
The Fold

The Fold IsnThe Fold Isn’’t Binary Eithert Binary Either
600 px
768 px
1024 px
Data courtesy of Data courtesy of ClickTaleClickTale
Free trial at Free trial at www.clicktale.comwww.clicktale.com
% of Users
The chrome steals
about 170 pixels

Analyze User Screen Height Distribution
to Select Design Height for Key Pages
1030, 5%
866, 25%
600, 100%
1024, 20%
1000, 20%
864, 29%
807, 29%
768, 92%
772, 38%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
600 650 700 750 800 850 900 950 1000 1050 1100
Monitor Screen Height (pixels)
%ofUserswithGiven(orHigher)ScreenHeight
Going above
768 px = big drop
Page Design Height (pixels)

Marrying UI Design with Metrics
Think of “Ease of use”
as distinct from the
functionality
User Interface
elements matter
Position
Layout
Size
Color
Font
Text copy
Interaction design
Navigation

Automated Multivariate Testing:
Google Website Optimizer

Case Studies:Case Studies:
Translating Metrics Into ActionTranslating Metrics Into Action

Case Study 1: Quicken BrokerageCase Study 1: Quicken Brokerage
Optimizing Sign In/Registration FlowOptimizing Sign In/Registration Flow
100%
62.3%
58.8%
50.9%
34.4%
32.7%
0%
20%
40%
60%
80%
100%
% of Users% of Users
Sign in /Sign in /
RegistrationRegistration
Account Account
TypeType
Cash vs.Cash vs.
MarginMargin
5 Partner5 Partner
PagesPages
3 Partner3 Partner
PagesPages
Biggest
drop

Open
Account
Sign in
Account
Selection
Register
56%
44%
Forget
Password
Registration
Process
45% drop off
(20% of total)
36% overall
drop off for
this step
70%
(32% of Total)
17% drop off
(10% of total)
20% drop off
(6% of total)
30%
(14% of Total)
80%
(26% of Total)
55%
(24% of Total)
64%
of Total
Mapping the Flow to See WhereMapping the Flow to See Where
Users Were Dropping OffUsers Were Dropping Off
Change
Password
83%
(46% of Total)

Redesigned User Flow Improved Redesigned User Flow Improved
Registration Conversion Rate 37%Registration Conversion Rate 37%
37% improvement
in conversion rate
Released
New Design

• Which metric has highest ROI opportunity?
Case Study 2: FriendsterCase Study 2: Friendster
Optimizing Viral GrowthOptimizing Viral Growth
Active
Users
Prospective
Users
Invite Click
Succeed
Invite
click-through rate
Conversion
rate
Don’t
Click
Fail
Invites per
sender
% of users
sending
invites
• Multiplied together, these metrics determine your viral ratio
Users
% of users
who are
active
= 15%
= 2.3
= 85%
Registration
Process

The Upside Potential of a Metric
0
100%
Registration
Process Yield
0
100%
% of users sending
invitations
0
?
Avg # of invites
sent per sender
2.3
85%
15%
Max possible
improvement
0.15 / 0.85 = 18% 0.85 / 0.15 = 570% ? / 2.3 = ?%

Okay, so how can we improve the metric?
How do we increase the average number of
invites being sent out per sender?
For each idea:
What’s the expected benefit? (how much will it
improve the metric?)
What’s the expected cost? (how many engineer‐
hours will it take?)
You want to identify highest benefit/cost idea

Case Study 2: FriendsterCase Study 2: Friendster
Doubled Number of Invitations Sent per SenderDoubled Number of Invitations Sent per Sender

ContinuousContinuous
ImprovementImprovement

Adding Metrics and Optimization to
your Product Process
PlanPlan
DesignDesign
DevelopDevelop
Business
Objectives
Product
Objectives
Prioritized
Feature List
Scoping
Requirements
& Design
Code Test Launch
Site Level
Feature
Level
OptimizeOptimize Metrics & User
Feedback

Optimization through Iteration:Optimization through Iteration:
Continuous ImprovementContinuous Improvement
Measure
the metric
Analyze
the metric
Identify top
opportunities
to improve
Design & develop
the enhancement
Launch the
enhancement
Learning
Gaining knowledge:
• Market
• Customer
• Domain
• Usability

Early Stage Product Management Early Stage Product Management
Cheat SheetCheat Sheet
Truly understand your customer needs
Ruthlessly prioritize and launch v1
Talk with users 1‐on‐1 and get feedback
Define equation of your business
Identify and track key metrics
Identify opportunities & prioritize by ROI
Launch, learn, and iterate

Questions?
@danolsen
dan@yourversion.com www.yourversion.com

Early stage-web-product-management-by-dan-olsen-090728040916-phpapp02

Recomendados

Recomendados

Más contenido relacionado

Último

Último (20)

Destacado

Destacado (20)

Early stage-web-product-management-by-dan-olsen-090728040916-phpapp02