Más contenido relacionado La actualidad más candente (20) Similar a Acing the A/B Test: How to Optimize to Send Extraordinary Emails (20) Acing the A/B Test: How to Optimize to Send Extraordinary Emails1. T I M O Z M I N A
Acing the A/B Test: How to Optimize to Send Extraordinary
Emails
2. 2Proprietary and Confidential | © Marketo, Inc.
• This webinar is being recorded, the slides and recording will be sent to you once the webinar
con clu des
• Qu estion s? Feel free to en ter th em in to th e Q&A box on y ou r screen an d we’ll get to th em at th e
en d of th e session
• Postin g to social? Please u se ou r h ash tag # m ktgn ation
Housekeeping
4. 4Proprietary and Confidential | © Marketo, Inc.
A Brief History of Emails
1971 –Firstemail was sent
1985 –Email is usedby government workers,military,
academics
1991 –Thefirstemail wassent fromspace
1996 –Microsoftand Hotmail releaseemail tools
1997 –Microsoftbuys Hotmail; almost 10 million
emailusers
1998 –Theterm“Spam” enters the dictionary
2003 –Over77 millionusers
2004 –U.S. Governmentstartsregulatingunsolicited
emails
2009 –94% ofallemails werespam
2019 –3.9 billionemailusers; 293 billionemails sent
per day; 64% ofemailis spam
Sources: MacWorld, 99firms, Radicati, EMarketer
5. 5Proprietary and Confidential | © Marketo, Inc.
Why Should You A/B Test?
What if you don’t test?
Youdon’tlearnwhatis workingandwhatisn’t
Youmissouton meaningfulengagements
Yourgraphsdon’ttrendup andto theright
Lossofrevenue
Youdon’tgrow
Whatifyou dotest?
You canbreakthroughthenoise of 293 billion emails sentper day
You canoptimize EVERYTHING
You canlet thedatado thetalking
You captureMORE meaningful engagements
Increaserevenue
You grow
6. 6Proprietary and Confidential | © Marketo, Inc.
Now We Know Why… But WhatIs the Process?
Define Your
Goal
Decide
What to
Test
Create a
Hypothesis
Test
Evaluate
Results
Make a
Decision
7. 7Proprietary and Confidential | © Marketo, Inc.
It’s Time to Pick a Test!
What should you test?
Subject line
Numberofwordsin SL
Personalizedvs. Non-personalized
Brackets
Call-to-actionbutton
Size
Color
Font
Italicizevs. Bold
Preheader
Tone of voice
HTML vs. Text email
Day of week
Time of day
Length of email
From name
And many, many more!
8. 8Proprietary and Confidential | © Marketo, Inc.
How Do You ActuallyRunthe Test?
A few things to keep in mind:
Onlytestone element atatime
Keep thetesting environment asclean aspossible
Send tothe same audience
Send at the same time
Gatherasmuch dataaspossible
VS
9. 9Proprietary and Confidential | © Marketo, Inc.
Before We Evaluate…
Statistical Significance:
anumberthatexpressestheprobabilitythattheresult ofagiven experimentorstudycouldhaveoccurredpurelybychance. Thisnumbercan
beamargin oferror(“Theresults ofthispublic opinionpoll areaccuratetofivepercent”), oritcan indicateaconfidencelevel (“If this
experimentwererepeated,thereis aprobabilityofninety-fivepercentthatourconclusionswould besubstantiated”).
10. 10Proprietary and Confidential | © Marketo, Inc.
Now It’s Time to Evaluate
Results:
Open ratesincreased by4% (100% statisticalsignificance)
Click-through-ratesincreasedby47% (100%statisticalsignificance)
Click-to-openratesincreasedby41% (100%statisticalsignificance)
(ThanksMike!)
What doesthis mean?
Contextis amazing…
Adding contextnotonlyincreased opens,butalsoincreased clicks
(ThanksMike!)
Specifically for this test:
Earlystageaudience
Marketingtitles
Engagedin thepast1year
11. 11Proprietary and Confidential | © Marketo, Inc.
And theWinner Is…
WhatdoI donow?
DOCUMENT YOUR FINDINGS!
Letyour teamknow
Include bracketsto thataudience
Testthison your otheraudiences (if you weren’ttestingitalready)
Pick anothertest
Takeit astepfurther
13. 13Proprietary and Confidential | © Marketo, Inc.
I KnowWhat You’re Thinking:
We just beat the control and now knowthat adding
brackets to the subject line can help with not only
opens,but also clicks… what else is there?
15. 15Proprietary and Confidential | © Marketo, Inc.
The ResultsLook Conclusive
Results:
Open ratesincreased byover 15% (100%statisticalsignificance)
Click-through-ratesincreasedbyalmost30% (100% statisticalsignificance)
Click-to-openratesincreasedbyover 12% (100% statisticalsignificance)
Looksconclusive, right?
18. 18Proprietary and Confidential | © Marketo, Inc.
Adding a Secondary Call-to-Action
We’vebeentoldit’sa no-no
Decreasesclicks
Increasesunsubscribes
Can hurtdeliverability
Have you ever testedit?
For thistest:
Senttoallbehaviorscoreaudiences
Marketingtitles
Over 450kemails
Engagedinpastoneyear
19. 19Proprietary and Confidential | © Marketo, Inc.
The Results
Results:
Open ratesincreased by0% (no statisticalsignificance)
Click-through-ratesdecreasedby3% (no statisticalsignificance)
Click-to-openratesdecreasedby3% (no statisticalsignificance)
What does this mean?
Wecannotconfidentlysaythataddinga secondCTA directlyincreases/decreases the
performanceofan email
Disclaimer:Trythisonyouraudience, don’tjust takemy wordforit!
20. 20Proprietary and Confidential | © Marketo, Inc.
5 Key Takeaways
Documentyour findings
Keepyour tests clean
Dig deeper
Find statisticalsignificance
Always be testing
Notas del editor Focused Investments
Track Record of Delivery
Innovation