As a Sitecore Experience Optimization partner - Conversionista presents how you can create unexpected and fantastic results through AB testing

1. Unleash the power of AB- testing
Digital marketing Summit 2016

2. The Optimisation
Wheel

3. This is where most start
“The most innovative
company in our
industry”
“Amazing creativity is
the core of everything
we do”

4. What’s missing?

5. We are data driven - Horay!

6. Add experiments! Done or?

7. What’s missing?

8. 8

9. 9

10. 10
This way or that
A / B testing direct visitors at random to
different versions of a page or a
component.
The experiment is performed on your
regular site on your real visitors and they
are not aware that they are in an
experiment.

11. 11
The classic A/B -test
A B

12. 12
We have a winner!
You measure the outcome against
the experiment goals (for example
sales) for the different variations.
When the test has reached statistical
significance a winner can be
declared.

13. 13
Överfralla eller Underfralla?

14. Are you a scientist or an accountant?
14

15. “Where the rubber hits the road”
A/B Testning is the
point when our
optimization wheel hits
the cruel reality.

16. Two things happen - both are important
Forward movement
Gravel spray

17. What to expect
You will see one of these three outcomes:
17
1. Winning
2.Losing
3.Indecisive

18. 18
# Winning tests
Total # of tests
Win rate =

19. 19
What’s a realistic goal?
50%0% 100%
In-house Agencies
19

20. What’s a bad test?
The only bad test is
the one you don’t
learn anything from
20

21. Case
-
Lifesum (ShapeUpClub)

22. The Original
22

23. Variation 1
23

24. Test 1
24

25. Results
25
Engagement
Conversions
Revenue
+96%
+ 12%
+ 20%
+21%
-15%
+11%
+141%

26. WTF?
“Don’t be afraid to prioritize”
26

27. Test 2
27

28. Results
28
Engagement
Conversions
Revenue
+ 153% (+96%)
+ 51% (+12%)
+ 67% (+ 20%)
+26%
+ 32%
+121%
+188%

29. Key takeaways
- The only failed test is the one you
don’t learn from
- Best practices don’t always work
- Use several data points and
segments

30. Case - Namnband

32. Conversions
Revenue
A/B-test #1
-6 %
-13 %

33. A/B-test
#2
+16 %
+15 %
-25 %
-48 %
Conversions
Revenue
Conversions
Revenue

34. 34

35. 35

36. Case - Spotify Premium

37. Premium trial for US market
High Dropoff rate
Original
You must leave your credit card
details in order to start your 30
day free trial
WHY?
37

38. Input to the hypothesis
We made a short exit survey.
The main reason stated:
38
“I don’t want to leave my
credit card details for
something that is free”!

39. Hypothesis:
Give the user “A reason why”
39
We only use this to verify your account, you
won't be charged anything for your trial
We need this just in case you decide to stay
Premium after your free month
We need this because our music deals only
allow free trials for users that are credit card or
PayPal holders
A
B
C

40. The result
40
Original
“We only use this to verify…”
Blame the record companies
“In case you decide to stay…”

41. 41

43. I’m waiting for the test results
Or my retirement
Whatever comes first

44. 44
Visual test duration calculator
apps.conversionista.se

45. Case - Software company

46. Promo 1
46

47. Original
47

48. Variation - By the book
48

49. The Results?
49
0%
Because visitors don’t
make any decision on
this page!

50. Average E-Commerce Uplift per Template Type
50

51. Testing sitewide to reduce bounce rate
51

52. Test 1 - Functional tagline
52
Engagement: + 6%
Original
Variation

53. Test 2 - Authority badge
53
Engagement: ±0%
Variation
Original

54. Product
page test
54

55. Case - Swedoffice - Stripped Checkout
55

56. Checkout test
56
Test # 2

57. 57
57
Remove navigation
Change up sales

58. The result?
58
58
0!

59. Why?
A heatmap in visual website optimizer gave us the answer
59

60. Test
#2
60
Conversion rate +6%
Revenue per visitor + 10%

61. The BEST global
speakers
We’ve sold out every
singer year
+ 600 Attendees
No sponsors on stage
Scandinavia’s #1
Conversion Event
conversionjam.se