Matt Lacey, Head of Optimization at PRWD, gave a presentation at #SASCon about exploring the full range of A/B testing opportunities. He discussed how conversion optimization is an essential element that goes beyond just testing, and provided examples of tests his company ran for clients that increased subscriptions by 12.5% by highlighting a savings and 77% by changing a subscription format. Lacey emphasized that A/B testing should serve the purpose of clearly testing business hypotheses.

1. @matt_lacey12 #SASCon
EXPLORING THE FULL RANGE OF
A/B TESTING OPPORTUNITY
Matt Lacey
Head of
Optimisation

2. @matt_lacey12 #SASCon
Team of leading conversion optimisation
specialists
Head office in Manchester, with clients
across UK and Europe
Work with brands to analyse, research, test
and optimise site experiences
ABOUT PRWD

3. @matt_lacey12 #SASCon
“We’ll make the big changes
and you can just do the
optimisation.”
“Conversion Optimisation &
Continuous Improvement are
part of our DNA.”

4. @matt_lacey12 #SASCon
CONVERSION
OPTIMISATION IS NOT
JUST TESTING ...
... BUT IT IS AN
ESSENTIAL ELEMENT.

5. @matt_lacey12 #SASCon
Four Options:
1. Control – (4 months FREE)
2. Variation A – (SAVE £22)
3. Variation B – (SAVE 38%)
4. Variation C – remove ‘(cancel
anytime)’ from monthly
WHICH TEST
WON?
12.5%overall increase in
subscriptions for Variation A (SAVE £22)

6. @matt_lacey12 #SASCon
1. Control - Annual £39.99
(Save £44)
2. Variation – £3.33 per
month (£39.99 annually)
WHICH TEST
WON?
77%overall increase in annual
subscriptions for the Variation

7. @matt_lacey12 #SASCon
“We had gotten really good at
optimising, tuning and iterating, but in
the process we had lost sight of the
purpose of those activities: testing a
clear business hypothesis in the
service of the company’s vision.”
Eric Ries, Author the Lean Startup

8. @matt_lacey12 #SASCon
GET REVISING
SIGN-UP TABLE
“
Price doesn’t put me off, although the
basic package is more than enough for
me.
I think the site has already got
everything I need.
Moderated user research participant

9. @matt_lacey12 #SASCon

10. @matt_lacey12 #SASCon
GET REVISING
SIGN-UP TABLE
185%increase in premium subscriptions
with a small drop in free subscriptions. This was seen to
be a good trade-off.

11. @matt_lacey12 #SASCon
UI
TWEAKING
BUSINESS
HYPOTHESIS
TESTING
OPPORTUNITY

12. @matt_lacey12 #SASCon
THANKS
FOR LISTENING
MATT LACEY
Head of Optimisation
@matt_lacey12