Test for Success: A Guide to A/B Testing on Emails
•Descargar como PPTX, PDF•
7 recomendaciones•20,411 vistas
Email marketing is a key component to any successful marketing strategy — and it's constantly evolving! That's why testing and optimizing your communications is just as important as the strategy itself. Join Marketo's Jessica Langensand and Optimizely's Allison Sparrow in this informative webinar on A/B testing ideas and best practices.
Recomendados
Recomendados
Más contenido relacionado
La actualidad más candente
La actualidad más candente (20)
Destacado
Destacado (20)
Similar a Test for Success: A Guide to A/B Testing on Emails
Similar a Test for Success: A Guide to A/B Testing on Emails (20)
Más de Marketo
Más de Marketo (20)
Último
Último (20)
Test for Success: A Guide to A/B Testing on Emails
- 1. Test for Success A Guide to A/B Testing on Emails & Landing Pages
- 2. #abtesting Our presenters Jessica Langensand Marketing Programs Manager Marketo Allison Sparrow Marketing Automation Manager Optimizely
- 3. #abtesting A few things first... ● Slides and recording emailed to you immediately after the webinar. ● Ask questions in the chat box & on Twitter! o Use #abtesting.
- 4. #abtesting What you’ll learn ● Why should you A/B test? ● Setting up your first (effective) A/B test ● Testing Ideas and Success Stories
- 5. #abtesting Why Should You Care About A/B tests?
- 6. #abtesting
- 7. #abtesting
- 8. #abtesting There are 4.1 Billion email accounts in existence. Source: The Radicati Group
- 9. In 2013, SMBs voted that email marketing was #2 most important marketing channel. #abtesting #1, having a website. Source: eMarketer
- 11. A/B Testing is the process of showing different versions of a site to visitors, in order to see which one better achieves your goals. #abtesting
- 12. #abtesting An A/B test is...
- 13. #abtesting
- 15. #abtesting One Component One Page/Email
- 16. #abtesting One Component One Page/Email One Split Audience (50/50)
- 17. #abtesting A B
- 18. #abtesting A B 43.4% more pre-orders
- 19. #abtesting Why should you care about A/B tests? ● Let the data do the work ● Build internal alignment ● Your website and email channels are invaluable (make the most of it) ● It’s easy, and cheap
- 20. What is an (effective) A/B #abtesting test?
- 21. #abtesting What is an (effective) A/B test? ● Identify your company’s goals ● Choose where to test ● Choose what to test ● State your hypothesis ● Track and report results
- 22. #abtesting What are my company’s goals?
- 23. #abtesting What are my company’s goals? ● Purchases/conversions ● Revenue per visitor ● Lead generation ● Pageviews
- 24. #abtesting Where to test? Welcome Page Product Page
- 25. #abtesting Where to test? ● Welcome email ● Home page ● Product page ● Contact Form page ● 30 day trial expiration email ● Download confirmation email ● Cancellation confirmation email
- 26. #abtesting What to test?
- 27. #abtesting What to test? ● Headline ● Subject Line ● Value Proposition ● CTA
- 28. #abtesting Create a Hypothesis “Visitors are getting distracted and feeling overwhelmed with too many competing elements on the page. Removing elements will reduce the clutter, focus user attention on article content and result in an increase in article pageviews.”
- 29. #abtesting State a Hypothesis ● Identify a theory about the specific user experience ● Define the metrics for success ● Define the quantifiable change in the key metric
- 30. #abtesting Keep a Record!
- 31. #abtesting How to Determine a Winner
- 32. #abtesting Email
- 33. #abtesting Statistical Significance Source: ZettaSphere.com
- 34. #abtesting Statistical Significance Source: ZettaSphere.com
- 36. #abtesting Significant? Source: SplitTestCalculator.com
- 38. #abtesting Format • Font • Text Size • Color • Branding • Images :)
- 39. #abtesting Subject Lines A B
- 40. #abtesting A Subject Lines + 33% click through rate B
- 41. Subject Lines ● Vary a Replicable Element [New Report] 2014 Marketing Automation Vendor Comparison vs. [Download] 2014 Marketing Automation Vendor Comparison #abtesting ● Complete Subject Line Test o Test on a small group, wait 24 hours and send winner to remainder of your list
- 42. #abtesting “From” Line “Scott Magri” + 4% opens +3% click-through vs. “Marketo Premium Content”
- 43. #abtesting Button Placement +6% CTR
- 46. #abtesting Banners 10% CTR 5% CTR
- 48. #abtesting Button Color +21% CTR
- 49. #abtesting A/B testing isn’t about what you like. It’s about what your audience responds to.
- 50. #abtesting
- 51. #abtesting Questions? Use Chat
- 52. #abtesting Thank you! Recording is on the way!
Notas del editor
- Jessica
- Jessica
- jessica
- Jessica
- Jessica
- Jessica - We’re marketers, not mind readers. The purpose of A/B testing is to help you determine which version of your email or website converts the best. It’s an inexpensive way to generate more leads, without having to send out more communications. Even minor changes can produce a huge impact - but more on that later.
- Jessica It’s harder than ever to get your message heard through all the noise - so you need to be as effective as you can. On any given day, the average customer will be exposed to over 3,000 media messages, they will pay attention to 52 of those messages and will positively remember 4. This is why we as marketers need to become even better at capturing the attention of our audience and hopefully become one of those 4 messages that they remember.
- Jessica
- Jessica
- Note that it is entirely possible, and natural to have more than one company objective. An example is when working
- Once you have identified your conversion metrics that you want to follow, you can identify how different pages on your site affect that objective. If you’re looking to increase certain purchase numbers, like Electronic Arts, they smartly chose to A/B test their product page. Another example is if you are looking to increase number of trials started within a certain product. Look for pages that have high traffic, and high impact. High traffic pages, with a lot of visitors, will ensure that you reach accurate and effective results from your test as quickly as possible. Pages that have high impact on your business are pages that are an integral part of your company’s goals.
- Some examples of high traffic/high impact pages are your home page, a product page, or a form page, like Contact Us or Request a demo. You could also look at the search results page, article page, or pricing page.
- Now that we have identified a crucial step in your conversion goal’s flow, we now can choose what component of the site or email that we want to test. With so many components to a single email or webpage that you can test, are there some components that are better to start with than others? When you are first starting out with A/B testing, a couple easy guidelines to get going is to reduce noise on your site or email, while increasing focus to your goal. This is exactly what Electronic Arts did in removing their hero image. By increasing focus to the true call to action (purchasing pre-orders), moving up the buy button further up the page, as well as reducing noise on their site, they were able to successfully increase purchase conversions! If you are getting started with Email A/B testing, a great first test would be to test the subject line, using the same principles: reduce noise and increase focus. A great example of a test we here at Optimizely did was test out our subject line by explicitly highlighting what type of content we were providing in our promotional email. In doing so, we increased the focus for our users, reduced marketing jargon noise in our subject header, and as a result increased click-through rates.
- While there are many components to test within your page/email, start with something that will increase focus to your end goal. We will get into a lot more examples of types of tests you can run, but here are some good components to get started with.
- Hypothesis-driven testing means that you start by identifying what experience you want to change and how it will affect your business results -- not just changing things and hoping for the best. If you’ve done the first three steps, creating a hypothesis is not difficult. And I’m not telling you to state your hypothesis because like the Secret, if you say it, and mean it, it will be true. Stating a hypothesis encourages you to define exactly what you are testing, and with what goals, so that you can stay on track, and successfully report on that hypothesis when the results come in. Whenever I launch a new program, the first piece of the puzzle I always tackle is: what are my reports going to look like? By identifying exactly how I want my reporting to look, I have a better and more strategic idea of how I want to implement this new program. Stating your hypothesis identifies the particular component that you are testing, as well as the exact conversion metric you want to improve, as well as the scope. That way, when you are reporting on this A/B test down the road, you know exactly what to look for, and whether the A/B test was success, leaving no room for error. On top of that, stating a hypothesis reinforces internal team alignment, by putting the cards on the table, and leaving no doubt exactly what is being tested and what goals are being tracked.
- Hypothesis-driven testing means that you start by identifying what experience you want to change and how it will affect your business results -- not just changing things and hoping for the best. If you’ve done the first three steps, creating a hypothesis is not difficult. And I’m not telling you to state your hypothesis because like the Secret, if you say it, and mean it, it will be true. Stating a hypothesis encourages you to define exactly what you are testing, and with what goals, so that you can stay on track, and successfully report on that hypothesis when the results come in. Whenever I launch a new program, the first piece of the puzzle I always tackle is: what are my reports going to look like? By identifying exactly how I want my reporting to look, I have a better and more strategic idea of how I want to implement this new program. Stating your hypothesis identifies the particular component that you are testing, as well as the exact conversion metric you want to improve, as well as the scope. That way, when you are reporting on this A/B test down the road, you know exactly what to look for, and whether the A/B test was success, leaving no room for error. On top of that, stating a hypothesis reinforces internal team alignment, by putting the cards on the table, and leaving no doubt exactly what is being tested and what goals are being tracked. Average order value will increase Lead forms submitted will increase Number of purchases will increase
- It takes time and statistical significance to declare a winner for an experiment If your variation didn’t win, that’s still valuable data If you’re not getting significant results, try making bigger changes or using a page that gets more traffic
- Jessica
- Most email clients that allow you to A/B test will aggregate the results like this screenshot here. This particular test was on the From Name like the test I mentioned before. The first email was sent from Marketo Premium Content while the bottom one used tokens that populated with the lead owners name. In this case, the metric we were basing success off of was open rate and as you can see here, the email sent from the lead owner had a 17.2% open rate, versus premium content which had a 15.4% open rate
- If you’re not comfortable with finding significance on your own in Excel there are free calculators online that you can use. This particular one is on ZettaSphere.com and all you have to do is plug in the values that you were tracking and your sample size and it will tell you how significant your test was
- Or if your test is not significant, it will tell you to modify your test or your sample size in order to reach a more significant conclusion.
- ~8-10 minutes total Subject Line [brackets] From line Button Copy Button Placement
- When it comes to A/B tests, there are so many different variables to manipulate. You can think about font, text size, color etc. We’ve tested using speaker headshots in webinar invites vs. not and found positive results for including them which is why you all probably got in invite that included my and Allison’s faces in it.
- Jessica
- There are lots of things you can test on emails and landing pages. After this webinar, we’ll be sending a link to the recording as well as this joint cheat sheet from Optimizely and Marketo on the 30 things you can start A/B testing to increase lead generation. Now we’ll move on to how to declare a winner.
- Allison
- Cara