A Product Manager's Guide to Building the Right Software in Less Time

A Product Manager’s Guide
to Building the Right
Software in Less Time
UserTesting Webinar Featuring Forrester Research
February 27, 2019

2
.
USERTESTING
Tanya Koshy
VP Product Management
UserTesting
Christopher Condo
Senior Analyst
Forrester Research

4© 2019 FORRESTER. REPRODUCTION PROHIBITED.
 Companies must transform to keep pace with digitally savvy,
restless customers.
 They should reposition their internal structure to better
capitalize on emerging technologies.
 Organizations must respond to a constant barrage of internal
and external disruptions in their respective industries.
Forrester’s predictions for 2019

Transforming the business to be customer focused and
deliver value directly to the customer via digital capabilities.
Digital Transformation:

OLD WAY: Digital Experiences Are Optimized By What’s
Easiest to Execute
?
…The Customer Experience Is An
Afterthought
Did I fill it out
correctly?
Am I getting a
refund?

NEW WAY: Experience is Optimized for the Customer
Back-end capability is exposed via modern architectures
and UIs to Enable Customer-Obsessed CX

Customer Experience: The complete end-to-end user
experience the customer has with your product.

Improving The
Customer
Experience Is
Business Critical

38%
Of developers
use indirect
measurements
(like defects) to
gauge success
Oops!
Something
went wrong.
Source: Forrester Analytics’ Business Technographics Global Developer Survey, 2018

Shutterstock.com 99566564
Maybe We
Need More
Automation?
Maybe We Need
More
Experimentation?

12© 2017 Forrester Research, Inc. Reproduction Prohibited
Customer led
Insights driven
Fast
Connected
Customer-obsessed DNA
FROM
Customer aware
Data rich
Perfect
Siloed
TO

Organizations
Need Product
Managers To
Blend Customer
Insights With
Business
Objectives To
Drive AD&D
Outcomes.

Dev Focuses on
Velocity and
Quality
PM Focuses on
Customer Experience
and Outcomes

A Product-
Centric Team
Has Both Core
Members And
Supporting
Members

Use Experimentation To Capture Customer Insights
And Drive Product Development Direction
Direct User
Testing
A/B Testing Feature
Flags

Direct User Testing
Direct User Testing

Characteristics Of Direct User Testing
› Direct user testing provided the input from real people encouraged
to speak their thoughts while using your application to help you
understand “Why” they prefer one version over another or uncover
issues.
› Components:
• Real people invited to participate in testing
• A lab or platform that allows users to be recorded as they test your app
• A well-thought-out experiment or hypothesis for users to test
• Note: A finished app is not required for subjects to provide meaningful results
Direct User Testing

Direct User Testing (DUT)
Source: “AD&D Pros: Stop Guessing And Start Experimenting”
Direct User Testing

A/B Multivariate Testing

Characteristics Of A/B Multivariate Testing
› A/B Multivariate Testing enables in-production testing of different
versions of an application (work flow, algorithms, search engines,
etc.) to determine a winning strategy within hours or days.
› Components:
• A/B testing logic embedded in code
• A system that controls which populations see Experience A vs. Experience B
• A statistical engine to determine the “winning experience”

Feature Flagging
Feature Flagging

Characteristics Of Feature Flagging
› Feature Flagging allows application code to be released in “off”
mode and managed with a dashboard enabling single-trunk
development that enables stabile releases.
› Components:
• If-Then-Else logic embedded in app code
• App or service that communicates value of the feature flag
• Administrative console for managing flags
• Ability to configure which user populations see each feature when switched
“on”
Feature Flagging

Feature Flagging
Users Feature Flag Code
Feature Flagging

“This allowed us to deploy
code and feel confident
that customers are seeing
the right features.”
So What Did Developers See After Experimenting?
“We get senior executives
engaged on the release
process because they can
see the data from the
experiments.”
“Now our product manager
can coordinate with
marketing, sales etc. until
they are ready to release.”
“Using feature flags does
helps the team release
faster by helping us push
continuous integration.”
“These tools force us to
align around the customer
experience.”
“We reduced the number
of dependencies, people
involved, and the stress of
making changes. It’s much
easier to iterate and
improve.”

Experimentation Results In Fast Time To Value
› Create a premise to narrow down choices
› Get the pros involved for high-impact changes
› Know that building your own experimentation platform can quickly
become impractical
› Don’t use experimentation as a substitute for tried and true testing
› Understand that experimentation creates technical debt

A Product Manager’s Guide
to Building the Right
Software in Less Time
Tanya Koshy
VP Product Management, UserTesting

29
Validate & experiment at every stage

30
Customer
Who is my
customer?
Problem
What is their
problem?
Solution
What’s the
best, lightest
solution I can
build for
them?
Key Product Questions

I have an
idea for a
problem
area
I want to
learn more
about my
customers’
needs/pains
in this area
Do people
actually
have the
problem I
think they
have?
Are people
currently
unsatisfied
with their
experience
in this area?
How do
people solve
their
problem
today?

32
How often are you doing X?
Describe your most recent experience doing
X. What worked & what didn’t?
What made doing X easy?
Needs Discovery
Interview
Questions
What are their pain points doing
X? Great way to inspire new ideas
for products / features to build.
What made doing X frustrating/hard?
What did you use to do X?
How similar was your most recent experience
to how you typically do X?
How satisfied are you with X, in general? 1 =
not at all, 5 = very

I have a better understanding of their overall experience and
their biggest needs/pain points.

Do people
understand the
purpose of the
solution? Do
they find it
valuable?
Does the
solution
actually solve
the problem?
Does it solve it
well?

35
Feature 1
1 - 2 sentence description
Feature 2
Feature 3
Template for Testing Product Ideas as a Landing Page
Your 1-Line Value Prop Here

36
Spend 1 - 2 mins reviewing the page, share
your thoughts out loud as you do.
What are your initial impressions?
In your own words, based on this page,
describe what this product does?
Great Interview
Questions
Going through the page section by section,
which one is most compelling to you? Why?
Which is least compelling to you? Why?
What must this product do, in addition to
anything you saw here, that would get you to
use it?
How likely would you be to use this product?

We have a fine-tuned idea for a solution that customers find valuable.

Is the app easy
to use? What is
the first-time
user
experience
like? Is it
compelling?
What do we
need to
improve?

We have validated that our solution is usable and have
identified new areas for further improvement.

Bring customers into every
stage of development
From needs discovery, to concept
& prototype validation, thru live
site usability. Test mobile apps,
websites, & real world
experiences.
Results Within Hours
Rich video results from your target
customers, answering the
questions you have. Make
confident decisions & share data to
get buy-in across your org.
Same-day Video Interviews
Live Conversation: Schedule
and recruit live interviews in
less than a day. Talk face to
face & share screens.
Your Exact Customer
Easily recruit your target audience
from UserTesting’s diverse, global
panel.
UserTesting

41
Questions?
USERTESTING
If you have any further questions please contact us at: webinars@usertesting.com

42
Thank You!
USERTESTING
If you have any further questions please contact us at: webinars@usertesting.com

A Product Manager's Guide to Building the Right Software in Less Time

Recomendados

Recomendados

Más contenido relacionado

Más de UserTesting

Más de UserTesting (20)

Último

Último (20)

A Product Manager's Guide to Building the Right Software in Less Time