Accelerate your growth by moving from simple A/B testing to building an advanced testing program. This session will help you learn how to build advanced A/B tests. With this session, you’ll get better at interpreting your test results, and find out the best approach to analyse failed or inconclusive experiment.

1. Scaling your
Testing Program
Vivek Vilatia
Senior Product Manager, VWO
Anshul Gurbaxani
Associate Product Manager, VWO
Prem Kujur
Customer Success Manager, VWO

2. Ask Questions on Twitter
#AskVWO

3. What is A/B testing
In the most simplest form, A/B
Testing is comparing two versions
of a web page to see which version
performs better for a defined
conversion goal

4. Type of Tests
A/B Test
Change any element or section
of a webpage and test both
versions.
Split URL Test
Test performances of separate
URLs if the landing pages are
significantly different.
Multivariate Test
Run tests on multiple variation
combinations together. Needs
sufficient traffic.

5. Graduating to
Advanced tests

6. Simple A/B tests Advanced A/B tests
➔ Great starting point
➔ Smaller teams
➔ Easy to setup
➔ Quick wins
➔ Test changes like copy,
arrangement of elements,
CTA colors etc
➔ Test advanced use cases with
greater impact
➔ Conversion optimization program
➔ Self-sufficient team
➔ The right tool-set
➔ Management buy-in
➔ Tons of perseverance

7. Audience Changes MetricsPages
Key steps in setting up your tests

8. Test pages | Defining a set of similar pages for your test
Asterix * in the URL matches pattern would replace the variable part of the URL
Represented by * in the URL matches pattern

9. Audience | Complex targeting rules
Visitors logged in on
your website
Are either from India or have their
preferred currency as INR

10. Changes | Code editor for complex changes

11. Changes | Code editor for complex changes

12. Changes | Conditional Test Activation

13. Changes | Conditional Test Activation
SignUp Form Type Form

14. Measuring the
right metrics

15. Measuring the right metrics
Micro Conversions Macro Conversions

16. Measuring the right metrics

17. Never make your tests
live without proper
sanity checks

18. Quick previews

19. Cross Browser testing

20. Live previews

21. Check cookies using a plugin

22. Making sense of your
A/B testing reports

23. Making sense of your A/B testing reports

24. Run your test for at least a week.
Along with a primary goal, have related secondary goals.
Ensure that your primary goal represents the direct and
exclusive impact of the changes being tested.
Making sense of your A/B testing reports - Best Practices

25. Slice and Dice your test results to learn the trends and
behaviour of various user segments.
Testing radical changes at times might just do the trick
Identify micro-conversions and important correlative metrics
increase in a given variation, implement it.
Inconclusive Tests

26. Ensure that you don’t use segments with very less data.
Make sure there is no error in the test setup.
Ensure that you are not running multiple tests
simultaneously in the same part of the journey you are A/B
testing
Inconclusive Tests - Pitfalls

27. Thank You!
Q&A