SlideShare una empresa de Scribd logo
1 de 87
Descargar para leer sin conexión
#brightonseo
Winner or Loser?
Analysing Your Test
Results with Causal
Impact on R Studio
Slideshare.Net/GiuliaPanozzo1
@SequinsNSearch
Giulia Panozzo
Giulia Panozzo - @SequinsNSearch #brightonseo
Pre-
implementation
data
Post-
implementation
data
Giulia Panozzo - @SequinsNSearch #brightonseo
Giulia Panozzo - @SequinsNSearch #brightonseo
Why do we care
about Causal Impact?
Giulia Panozzo - @SequinsNSearch #brightonseo
Because Testing is Hard!
Giulia Panozzo - @SequinsNSearch #brightonseo
Giulia Panozzo - @SequinsNSearch #brightonseo
Giulia Panozzo - @SequinsNSearch #brightonseo
Giulia Panozzo - @SequinsNSearch #brightonseo
Example
‘Let’s add a price point to the title tag!’
Giulia Panozzo - @SequinsNSearch #brightonseo
Example
‘Let’s add a price point to the title tag!’
Giulia Panozzo - @SequinsNSearch #brightonseo
‘But will this help
bring more traffic
in?’
Giulia Panozzo - @SequinsNSearch #brightonseo
‘Well…’
Giulia Panozzo - @SequinsNSearch #brightonseo
Giulia Panozzo - @SequinsNSearch #brightonseo
However, with Causal Impact…
X
Giulia Panozzo - @SequinsNSearch #brightonseo
Causal Impact gives you the
confidence to leverage
statistically significant results and
drive changes at scale
Giulia Panozzo - @SequinsNSearch #brightonseo
‘But will this help
bring more traffic
in?’
Giulia Panozzo - @SequinsNSearch #brightonseo
YES
(Most likely)
NO
(Most likely)
Giulia Panozzo - @SequinsNSearch #brightonseo
You can use Causal Impact on a
number of domains, not only on
SEO tests!
Giulia Panozzo - @SequinsNSearch #brightonseo
Impact of feature changes on
app installs
Giulia Panozzo - @SequinsNSearch #brightonseo
Impact of influencer campaigns
Giulia Panozzo - @SequinsNSearch #brightonseo
Impact of offline events and
campaigns
Giulia Panozzo - @SequinsNSearch #brightonseo
And almost any time series data
Giulia Panozzo - @SequinsNSearch #brightonseo
If you run Causal Impact on R
Studio,
it’s free and open-source
Giulia Panozzo - @SequinsNSearch #brightonseo
Giulia Panozzo - @SequinsNSearch #brightonseo
What is Causal Impact
And how it can help your
strategy
Giulia Panozzo - @SequinsNSearch #brightonseo
What is Causal Impact?
Giulia Panozzo - @SequinsNSearch #brightonseo
Powerful package to analyse
data and infer the
cumulative impact of a change
in a time series
Giulia Panozzo - @SequinsNSearch #brightonseo
Based on BSTS
(Bayesian Structural Time
Series)
statistical model
Giulia Panozzo - @SequinsNSearch #brightonseo
Uses past data to predict the
outcome in the absence of the
treatment
(the counterfactual)
Giulia Panozzo - @SequinsNSearch #brightonseo
Defines the impact by
measuring the deviation of the
actual VS predicted outcome
Giulia Panozzo - @SequinsNSearch #brightonseo
What is Causal Impact?
Giulia Panozzo - @SequinsNSearch #brightonseo
How can it help us in marketing?
Giulia Panozzo - @SequinsNSearch #brightonseo
How can it help us in marketing?
Example of a clear winner from a title tag change
Clicks: +58% CTR: +38% Position: -15%
Giulia Panozzo - @SequinsNSearch #brightonseo
It can validate proposed strategy
changes in case of any doubts
Giulia Panozzo - @SequinsNSearch #brightonseo
It’s great to clearly show
stakeholders
the impact of our team’s work
Giulia Panozzo - @SequinsNSearch #brightonseo
It can help forecast the
direction of changes at scale and
help
make a case for more resources
Giulia Panozzo - @SequinsNSearch #brightonseo
How can it help us in marketing?
Example of a clear loser from a title tag change
Giulia Panozzo - @SequinsNSearch #brightonseo
By clearly identifying a winner or
loser, we can understand
what works and doesn’t work
for our audience
Giulia Panozzo - @SequinsNSearch #brightonseo
How to run your analysis
with Causal Impact on
R Studio
Giulia Panozzo - @SequinsNSearch #brightonseo
1. Download R Studio
Download R first
https://cran.r-project.org/
Download RStudio
https://www.rstudio.com/products/rstudio/download/
Giulia Panozzo - @SequinsNSearch #brightonseo
1. Download R Studio
Giulia Panozzo - @SequinsNSearch #brightonseo
1.1 Install Causal Impact
Giulia Panozzo - @SequinsNSearch #brightonseo
2. Prepare the data
Giulia Panozzo - @SequinsNSearch #brightonseo
2. Prepare the data
Giulia Panozzo - @SequinsNSearch #brightonseo
The first column is always
your test group.
Other columns can be used as
control groups if they are a good
fit
Giulia Panozzo - @SequinsNSearch #brightonseo
The pre-period should be at least
twice as long as the post-period,
to allow the model to plot the
actual VS predicted outcome
Giulia Panozzo - @SequinsNSearch #brightonseo
Any column with 0 should be
either removed or corrected
Giulia Panozzo - @SequinsNSearch #brightonseo
Isolated 0 in data set
Multiple 0s
VS
Giulia Panozzo - @SequinsNSearch #brightonseo
3. Run the script!
Giulia Panozzo - @SequinsNSearch #brightonseo
Choose file to import
Giulia Panozzo - @SequinsNSearch #brightonseo
Check preview
Giulia Panozzo - @SequinsNSearch #brightonseo
Set pre and post
periods
Giulia Panozzo - @SequinsNSearch #brightonseo
It’s a winner!
Giulia Panozzo - @SequinsNSearch #brightonseo
Giulia Panozzo - @SequinsNSearch #brightonseo
Giulia Panozzo - @SequinsNSearch #brightonseo
Now give it a go!
Request access to this
script here
Giulia Panozzo - @SequinsNSearch #brightonseo
What I’ve learned from (several)
trials and errors…
Giulia Panozzo - @SequinsNSearch #brightonseo
The date column should always
be removed
when using this script
Giulia Panozzo - @SequinsNSearch #brightonseo
Column titles can error out if
they contain special characters,
spaces, capitalised letters
Giulia Panozzo - @SequinsNSearch #brightonseo
Start small, then expand your
datasets with additional controls
and features once you’re
comfortable with the script
Giulia Panozzo - @SequinsNSearch #brightonseo
Statistics VS Everything
Else
Limitations & Takeaways
Giulia Panozzo - @SequinsNSearch #brightonseo
1. External events can impact
your data
Giulia Panozzo - @SequinsNSearch #brightonseo
Google algo
updates
Giulia Panozzo - @SequinsNSearch #brightonseo
Tools
tracking
failures
Giulia Panozzo - @SequinsNSearch #brightonseo
Engineering
releases
Giulia Panozzo - @SequinsNSearch #brightonseo
Create a document to map
internal changes & external events
This will allow you to take into account any
other known factors and isolate the treatment
in the analysis
Giulia Panozzo - @SequinsNSearch #brightonseo
2. Mind the Outliers!
Giulia Panozzo - @SequinsNSearch #brightonseo
Giulia Panozzo - @SequinsNSearch #brightonseo
Outliers can
originate from…
Giulia Panozzo - @SequinsNSearch #brightonseo
New product launches
(within the test group)
Giulia Panozzo - @SequinsNSearch #brightonseo
Holidays and seasonal events
Giulia Panozzo - @SequinsNSearch #brightonseo
Results only from one page
Page 1
Page 2
Page 3
Page 4
Page 5
Giulia Panozzo - @SequinsNSearch #brightonseo
Tracking bugs
Giulia Panozzo - @SequinsNSearch #brightonseo
You can spot outliers and rule them out by…
Giulia Panozzo - @SequinsNSearch #brightonseo
Increasing the size of the test group
Giulia Panozzo - @SequinsNSearch #brightonseo
Adding control groups
Giulia Panozzo - @SequinsNSearch #brightonseo
Pre-processing your raw data
Giulia Panozzo - @SequinsNSearch #brightonseo
Giulia Panozzo - @SequinsNSearch #brightonseo
Giulia Panozzo - @SequinsNSearch #brightonseo
Giulia Panozzo - @SequinsNSearch #brightonseo
3. Beware of confirmation bias
Giulia Panozzo - @SequinsNSearch #brightonseo
Sometimes, your test will be inconclusive, or might
be a loser even when you thought it’d be an easy
winner
Giulia Panozzo - @SequinsNSearch #brightonseo
Giulia Panozzo - @SequinsNSearch #brightonseo
In that case, you can run the test a little longer,
or repeat the test with bigger groups
Giulia Panozzo - @SequinsNSearch #brightonseo
If it’s still inconclusive or a loser, it’s probably best
to revert the change and focus on other tests
Giulia Panozzo - @SequinsNSearch #brightonseo
References and useful resources
• How we use causal impact analysis to validate campaign success - Part and Sum
• Measuring No-ID Campaigns with Causal Impact - Remerge & Alicia Horsch
• Causal Impact – Data Skeptic
• R Studio on GitHub
• The Comprehensive R Archive Network
• Causal Impact for App Store Analysis - William Martin
#brightonseo
Thank You for
Listening!
Slideshare.Net/GiuliaPanozzo1
@SequinsNSearch
Giulia Panozzo

Más contenido relacionado

La actualidad más candente

Core Web Vitals Audit - Sophie Gibson - PDF - BrightonSEO.pdf
Core Web Vitals Audit - Sophie Gibson - PDF - BrightonSEO.pdfCore Web Vitals Audit - Sophie Gibson - PDF - BrightonSEO.pdf
Core Web Vitals Audit - Sophie Gibson - PDF - BrightonSEO.pdfSophie Gibson
 
Data-driven SEO & content strategy to reduce your customer acquisition costs
Data-driven SEO & content strategy to reduce your customer acquisition costsData-driven SEO & content strategy to reduce your customer acquisition costs
Data-driven SEO & content strategy to reduce your customer acquisition costsadlift
 
Brighton SEO 2022: How to ideate to land links in publications that really im...
Brighton SEO 2022: How to ideate to land links in publications that really im...Brighton SEO 2022: How to ideate to land links in publications that really im...
Brighton SEO 2022: How to ideate to land links in publications that really im...EvaCheng23
 
Giulia Panozzo | Neuroscience of Search | BrightonSEO April 2023
Giulia Panozzo | Neuroscience of Search | BrightonSEO April 2023Giulia Panozzo | Neuroscience of Search | BrightonSEO April 2023
Giulia Panozzo | Neuroscience of Search | BrightonSEO April 2023GiuliaPanozzo1
 
Why your tech optimisations are still sat in the backlog
Why your tech optimisations are still sat in the backlogWhy your tech optimisations are still sat in the backlog
Why your tech optimisations are still sat in the backlogVicky481083
 
How to Combat SERP Volatility - Adriana Stein - BrightonSEO Slides 2023pdf
How to Combat SERP Volatility - Adriana Stein - BrightonSEO Slides 2023pdfHow to Combat SERP Volatility - Adriana Stein - BrightonSEO Slides 2023pdf
How to Combat SERP Volatility - Adriana Stein - BrightonSEO Slides 2023pdfAS Marketing
 
Monet BrightonSEO Slides 2022
Monet BrightonSEO Slides 2022Monet BrightonSEO Slides 2022
Monet BrightonSEO Slides 2022MonetBlake
 
Agile SEO: Prioritise SEO Activities with Cadence and Risk Radius
Agile SEO: Prioritise SEO Activities with Cadence and Risk RadiusAgile SEO: Prioritise SEO Activities with Cadence and Risk Radius
Agile SEO: Prioritise SEO Activities with Cadence and Risk RadiusParth Suba
 
The Hidden Gems of Low search volume
The Hidden Gems of Low search volumeThe Hidden Gems of Low search volume
The Hidden Gems of Low search volumeLiraz Postan
 
The Value of Featured Snippets (BrightonSEO 2023).pdf
The Value of Featured Snippets (BrightonSEO 2023).pdfThe Value of Featured Snippets (BrightonSEO 2023).pdf
The Value of Featured Snippets (BrightonSEO 2023).pdfNiki Mosier
 
Martin McGarry - SEO strategy c/o England manager Gareth Southgate
Martin McGarry - SEO strategy c/o England manager Gareth SouthgateMartin McGarry - SEO strategy c/o England manager Gareth Southgate
Martin McGarry - SEO strategy c/o England manager Gareth SouthgateMartin McGarry
 
Data Driven Approach to Scale SEO at BrightonSEO 2023
Data Driven Approach to Scale SEO at BrightonSEO 2023Data Driven Approach to Scale SEO at BrightonSEO 2023
Data Driven Approach to Scale SEO at BrightonSEO 2023Nitin Manchanda
 
Content Design & its Role in SEO and Accessibility [BrightonSEO Spring 2023]
Content Design & its Role in SEO and Accessibility [BrightonSEO Spring 2023]Content Design & its Role in SEO and Accessibility [BrightonSEO Spring 2023]
Content Design & its Role in SEO and Accessibility [BrightonSEO Spring 2023]Chloe Smith
 
Brighton SEO April 2023 - Basic Automation Hacks For SEO
Brighton SEO April 2023 - Basic Automation Hacks For SEOBrighton SEO April 2023 - Basic Automation Hacks For SEO
Brighton SEO April 2023 - Basic Automation Hacks For SEOTom Pool
 
HELP! I've Been Hit By An Algorithm Update - Jess Maloney - BrightonSEO Apri...
HELP! I've Been Hit By An Algorithm Update - Jess Maloney - BrightonSEO  Apri...HELP! I've Been Hit By An Algorithm Update - Jess Maloney - BrightonSEO  Apri...
HELP! I've Been Hit By An Algorithm Update - Jess Maloney - BrightonSEO Apri...Jessica Maloney
 
Vanda Pokecz - brightonSEO presentation.pptx
Vanda Pokecz  - brightonSEO presentation.pptxVanda Pokecz  - brightonSEO presentation.pptx
Vanda Pokecz - brightonSEO presentation.pptxVanda Pokecz
 
How to Implement Machine Learning in Your Internal Linking Audit - Lazarina S...
How to Implement Machine Learning in Your Internal Linking Audit - Lazarina S...How to Implement Machine Learning in Your Internal Linking Audit - Lazarina S...
How to Implement Machine Learning in Your Internal Linking Audit - Lazarina S...LazarinaStoyanova
 
BrightonSEO October 2022 - Log File Analysis - Steven van Vessum.pdf
BrightonSEO October 2022 - Log File Analysis - Steven van Vessum.pdfBrightonSEO October 2022 - Log File Analysis - Steven van Vessum.pdf
BrightonSEO October 2022 - Log File Analysis - Steven van Vessum.pdfSteven van Vessum
 
Brighton SEO - Luis Bueno Tabernero - How to do an ASO Audit like in the 90's...
Brighton SEO - Luis Bueno Tabernero - How to do an ASO Audit like in the 90's...Brighton SEO - Luis Bueno Tabernero - How to do an ASO Audit like in the 90's...
Brighton SEO - Luis Bueno Tabernero - How to do an ASO Audit like in the 90's...Luis
 
BrightonSEO 2022.pdf
BrightonSEO 2022.pdfBrightonSEO 2022.pdf
BrightonSEO 2022.pdfIlia Markov
 

La actualidad más candente (20)

Core Web Vitals Audit - Sophie Gibson - PDF - BrightonSEO.pdf
Core Web Vitals Audit - Sophie Gibson - PDF - BrightonSEO.pdfCore Web Vitals Audit - Sophie Gibson - PDF - BrightonSEO.pdf
Core Web Vitals Audit - Sophie Gibson - PDF - BrightonSEO.pdf
 
Data-driven SEO & content strategy to reduce your customer acquisition costs
Data-driven SEO & content strategy to reduce your customer acquisition costsData-driven SEO & content strategy to reduce your customer acquisition costs
Data-driven SEO & content strategy to reduce your customer acquisition costs
 
Brighton SEO 2022: How to ideate to land links in publications that really im...
Brighton SEO 2022: How to ideate to land links in publications that really im...Brighton SEO 2022: How to ideate to land links in publications that really im...
Brighton SEO 2022: How to ideate to land links in publications that really im...
 
Giulia Panozzo | Neuroscience of Search | BrightonSEO April 2023
Giulia Panozzo | Neuroscience of Search | BrightonSEO April 2023Giulia Panozzo | Neuroscience of Search | BrightonSEO April 2023
Giulia Panozzo | Neuroscience of Search | BrightonSEO April 2023
 
Why your tech optimisations are still sat in the backlog
Why your tech optimisations are still sat in the backlogWhy your tech optimisations are still sat in the backlog
Why your tech optimisations are still sat in the backlog
 
How to Combat SERP Volatility - Adriana Stein - BrightonSEO Slides 2023pdf
How to Combat SERP Volatility - Adriana Stein - BrightonSEO Slides 2023pdfHow to Combat SERP Volatility - Adriana Stein - BrightonSEO Slides 2023pdf
How to Combat SERP Volatility - Adriana Stein - BrightonSEO Slides 2023pdf
 
Monet BrightonSEO Slides 2022
Monet BrightonSEO Slides 2022Monet BrightonSEO Slides 2022
Monet BrightonSEO Slides 2022
 
Agile SEO: Prioritise SEO Activities with Cadence and Risk Radius
Agile SEO: Prioritise SEO Activities with Cadence and Risk RadiusAgile SEO: Prioritise SEO Activities with Cadence and Risk Radius
Agile SEO: Prioritise SEO Activities with Cadence and Risk Radius
 
The Hidden Gems of Low search volume
The Hidden Gems of Low search volumeThe Hidden Gems of Low search volume
The Hidden Gems of Low search volume
 
The Value of Featured Snippets (BrightonSEO 2023).pdf
The Value of Featured Snippets (BrightonSEO 2023).pdfThe Value of Featured Snippets (BrightonSEO 2023).pdf
The Value of Featured Snippets (BrightonSEO 2023).pdf
 
Martin McGarry - SEO strategy c/o England manager Gareth Southgate
Martin McGarry - SEO strategy c/o England manager Gareth SouthgateMartin McGarry - SEO strategy c/o England manager Gareth Southgate
Martin McGarry - SEO strategy c/o England manager Gareth Southgate
 
Data Driven Approach to Scale SEO at BrightonSEO 2023
Data Driven Approach to Scale SEO at BrightonSEO 2023Data Driven Approach to Scale SEO at BrightonSEO 2023
Data Driven Approach to Scale SEO at BrightonSEO 2023
 
Content Design & its Role in SEO and Accessibility [BrightonSEO Spring 2023]
Content Design & its Role in SEO and Accessibility [BrightonSEO Spring 2023]Content Design & its Role in SEO and Accessibility [BrightonSEO Spring 2023]
Content Design & its Role in SEO and Accessibility [BrightonSEO Spring 2023]
 
Brighton SEO April 2023 - Basic Automation Hacks For SEO
Brighton SEO April 2023 - Basic Automation Hacks For SEOBrighton SEO April 2023 - Basic Automation Hacks For SEO
Brighton SEO April 2023 - Basic Automation Hacks For SEO
 
HELP! I've Been Hit By An Algorithm Update - Jess Maloney - BrightonSEO Apri...
HELP! I've Been Hit By An Algorithm Update - Jess Maloney - BrightonSEO  Apri...HELP! I've Been Hit By An Algorithm Update - Jess Maloney - BrightonSEO  Apri...
HELP! I've Been Hit By An Algorithm Update - Jess Maloney - BrightonSEO Apri...
 
Vanda Pokecz - brightonSEO presentation.pptx
Vanda Pokecz  - brightonSEO presentation.pptxVanda Pokecz  - brightonSEO presentation.pptx
Vanda Pokecz - brightonSEO presentation.pptx
 
How to Implement Machine Learning in Your Internal Linking Audit - Lazarina S...
How to Implement Machine Learning in Your Internal Linking Audit - Lazarina S...How to Implement Machine Learning in Your Internal Linking Audit - Lazarina S...
How to Implement Machine Learning in Your Internal Linking Audit - Lazarina S...
 
BrightonSEO October 2022 - Log File Analysis - Steven van Vessum.pdf
BrightonSEO October 2022 - Log File Analysis - Steven van Vessum.pdfBrightonSEO October 2022 - Log File Analysis - Steven van Vessum.pdf
BrightonSEO October 2022 - Log File Analysis - Steven van Vessum.pdf
 
Brighton SEO - Luis Bueno Tabernero - How to do an ASO Audit like in the 90's...
Brighton SEO - Luis Bueno Tabernero - How to do an ASO Audit like in the 90's...Brighton SEO - Luis Bueno Tabernero - How to do an ASO Audit like in the 90's...
Brighton SEO - Luis Bueno Tabernero - How to do an ASO Audit like in the 90's...
 
BrightonSEO 2022.pdf
BrightonSEO 2022.pdfBrightonSEO 2022.pdf
BrightonSEO 2022.pdf
 

Similar a Giulia Panozzo | BrightonSEO Measurefest | October 2022

Benchmark_Winner or Loser_GiuliaPanozzo.pdf
Benchmark_Winner or Loser_GiuliaPanozzo.pdfBenchmark_Winner or Loser_GiuliaPanozzo.pdf
Benchmark_Winner or Loser_GiuliaPanozzo.pdfGiuliaPanozzo1
 
SearchLeeds, Arianne Donoghue 'PPC hacks to help you optimise like a boss'
SearchLeeds, Arianne Donoghue 'PPC hacks to help you optimise like a boss' SearchLeeds, Arianne Donoghue 'PPC hacks to help you optimise like a boss'
SearchLeeds, Arianne Donoghue 'PPC hacks to help you optimise like a boss' Branded3
 
Keyword Research Tactics and Tools - BrightonSEO 2017
Keyword Research Tactics and Tools - BrightonSEO 2017Keyword Research Tactics and Tools - BrightonSEO 2017
Keyword Research Tactics and Tools - BrightonSEO 2017Stacey MacNaught
 
How To Monitor Your Page Speed At Scale With PageSpeed API
How To Monitor Your Page Speed At Scale With PageSpeed APIHow To Monitor Your Page Speed At Scale With PageSpeed API
How To Monitor Your Page Speed At Scale With PageSpeed APIozanketenci
 
Inbound PR - Building Links with Less Cold Outreach
Inbound PR - Building Links with Less Cold OutreachInbound PR - Building Links with Less Cold Outreach
Inbound PR - Building Links with Less Cold OutreachStacey MacNaught
 
Start-Up Techniques for Journalists and Newsrooms
Start-Up Techniques for Journalists and NewsroomsStart-Up Techniques for Journalists and Newsrooms
Start-Up Techniques for Journalists and NewsroomsEzra Eeman
 
PPC Hacks To Help You Optimise Like A Boss - Search Leeds - Arianne Donoghue
PPC Hacks To Help You Optimise Like A Boss - Search Leeds - Arianne DonoghuePPC Hacks To Help You Optimise Like A Boss - Search Leeds - Arianne Donoghue
PPC Hacks To Help You Optimise Like A Boss - Search Leeds - Arianne DonoghueArianne Donoghue
 
Pubcon Austin - AdWords Competitive Analysis
Pubcon Austin - AdWords Competitive AnalysisPubcon Austin - AdWords Competitive Analysis
Pubcon Austin - AdWords Competitive AnalysisSusan Wenograd
 
How to create an opportunity-first approach to digital PR for more consistent...
How to create an opportunity-first approach to digital PR for more consistent...How to create an opportunity-first approach to digital PR for more consistent...
How to create an opportunity-first approach to digital PR for more consistent...JBH - The Digital PR Agency
 
Copie de Yellow, Black and White 3D Illustrative How to Code Workshop Present...
Copie de Yellow, Black and White 3D Illustrative How to Code Workshop Present...Copie de Yellow, Black and White 3D Illustrative How to Code Workshop Present...
Copie de Yellow, Black and White 3D Illustrative How to Code Workshop Present...Darifa1
 
Pink and Green Aesthetic School Presentation Design.pptx
Pink and Green Aesthetic School Presentation Design.pptxPink and Green Aesthetic School Presentation Design.pptx
Pink and Green Aesthetic School Presentation Design.pptxYustinusJefriManguns
 
PubCon NOLA Online Marketing Tools
PubCon NOLA Online Marketing Tools PubCon NOLA Online Marketing Tools
PubCon NOLA Online Marketing Tools Ashley Segura
 
Get Your Ticket Effective zero-volume keyword research and why it’s important
 Get Your Ticket Effective zero-volume keyword research and why it’s important Get Your Ticket Effective zero-volume keyword research and why it’s important
Get Your Ticket Effective zero-volume keyword research and why it’s importantMarkWilliamsCook1
 
Utilising Search Console For Quick SEO Wins - BrightonSEO April 2017
Utilising Search Console For Quick SEO Wins - BrightonSEO April 2017Utilising Search Console For Quick SEO Wins - BrightonSEO April 2017
Utilising Search Console For Quick SEO Wins - BrightonSEO April 2017Laura Hogan
 
BoSON22 | Janna Bastow | From Vanity Metrics to Effective Product Culture
BoSON22 | Janna Bastow | From Vanity Metrics to Effective Product CultureBoSON22 | Janna Bastow | From Vanity Metrics to Effective Product Culture
BoSON22 | Janna Bastow | From Vanity Metrics to Effective Product CultureBusiness of Software Conference
 
Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018)
Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018)Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018)
Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018)Matthew Philip
 
Self Care Green and Pink Newsletter Elegant Minimal Pastel Presentation.pptx
Self Care Green and Pink Newsletter Elegant Minimal Pastel Presentation.pptxSelf Care Green and Pink Newsletter Elegant Minimal Pastel Presentation.pptx
Self Care Green and Pink Newsletter Elegant Minimal Pastel Presentation.pptxLovekushKumar12
 
How to inspire content and dev teams to get the results in search you need.
How to inspire content and dev teams to get the results in search you need.How to inspire content and dev teams to get the results in search you need.
How to inspire content and dev teams to get the results in search you need.Steven Wilson-Beales
 
SearchLove San Diego 2019 - Vince Nero - Creating Great Content in Boring Ind...
SearchLove San Diego 2019 - Vince Nero - Creating Great Content in Boring Ind...SearchLove San Diego 2019 - Vince Nero - Creating Great Content in Boring Ind...
SearchLove San Diego 2019 - Vince Nero - Creating Great Content in Boring Ind...Distilled
 
Holiday catalog testing with Christopher Werler
Holiday catalog testing with Christopher WerlerHoliday catalog testing with Christopher Werler
Holiday catalog testing with Christopher WerlerSarah Fletcher
 

Similar a Giulia Panozzo | BrightonSEO Measurefest | October 2022 (20)

Benchmark_Winner or Loser_GiuliaPanozzo.pdf
Benchmark_Winner or Loser_GiuliaPanozzo.pdfBenchmark_Winner or Loser_GiuliaPanozzo.pdf
Benchmark_Winner or Loser_GiuliaPanozzo.pdf
 
SearchLeeds, Arianne Donoghue 'PPC hacks to help you optimise like a boss'
SearchLeeds, Arianne Donoghue 'PPC hacks to help you optimise like a boss' SearchLeeds, Arianne Donoghue 'PPC hacks to help you optimise like a boss'
SearchLeeds, Arianne Donoghue 'PPC hacks to help you optimise like a boss'
 
Keyword Research Tactics and Tools - BrightonSEO 2017
Keyword Research Tactics and Tools - BrightonSEO 2017Keyword Research Tactics and Tools - BrightonSEO 2017
Keyword Research Tactics and Tools - BrightonSEO 2017
 
How To Monitor Your Page Speed At Scale With PageSpeed API
How To Monitor Your Page Speed At Scale With PageSpeed APIHow To Monitor Your Page Speed At Scale With PageSpeed API
How To Monitor Your Page Speed At Scale With PageSpeed API
 
Inbound PR - Building Links with Less Cold Outreach
Inbound PR - Building Links with Less Cold OutreachInbound PR - Building Links with Less Cold Outreach
Inbound PR - Building Links with Less Cold Outreach
 
Start-Up Techniques for Journalists and Newsrooms
Start-Up Techniques for Journalists and NewsroomsStart-Up Techniques for Journalists and Newsrooms
Start-Up Techniques for Journalists and Newsrooms
 
PPC Hacks To Help You Optimise Like A Boss - Search Leeds - Arianne Donoghue
PPC Hacks To Help You Optimise Like A Boss - Search Leeds - Arianne DonoghuePPC Hacks To Help You Optimise Like A Boss - Search Leeds - Arianne Donoghue
PPC Hacks To Help You Optimise Like A Boss - Search Leeds - Arianne Donoghue
 
Pubcon Austin - AdWords Competitive Analysis
Pubcon Austin - AdWords Competitive AnalysisPubcon Austin - AdWords Competitive Analysis
Pubcon Austin - AdWords Competitive Analysis
 
How to create an opportunity-first approach to digital PR for more consistent...
How to create an opportunity-first approach to digital PR for more consistent...How to create an opportunity-first approach to digital PR for more consistent...
How to create an opportunity-first approach to digital PR for more consistent...
 
Copie de Yellow, Black and White 3D Illustrative How to Code Workshop Present...
Copie de Yellow, Black and White 3D Illustrative How to Code Workshop Present...Copie de Yellow, Black and White 3D Illustrative How to Code Workshop Present...
Copie de Yellow, Black and White 3D Illustrative How to Code Workshop Present...
 
Pink and Green Aesthetic School Presentation Design.pptx
Pink and Green Aesthetic School Presentation Design.pptxPink and Green Aesthetic School Presentation Design.pptx
Pink and Green Aesthetic School Presentation Design.pptx
 
PubCon NOLA Online Marketing Tools
PubCon NOLA Online Marketing Tools PubCon NOLA Online Marketing Tools
PubCon NOLA Online Marketing Tools
 
Get Your Ticket Effective zero-volume keyword research and why it’s important
 Get Your Ticket Effective zero-volume keyword research and why it’s important Get Your Ticket Effective zero-volume keyword research and why it’s important
Get Your Ticket Effective zero-volume keyword research and why it’s important
 
Utilising Search Console For Quick SEO Wins - BrightonSEO April 2017
Utilising Search Console For Quick SEO Wins - BrightonSEO April 2017Utilising Search Console For Quick SEO Wins - BrightonSEO April 2017
Utilising Search Console For Quick SEO Wins - BrightonSEO April 2017
 
BoSON22 | Janna Bastow | From Vanity Metrics to Effective Product Culture
BoSON22 | Janna Bastow | From Vanity Metrics to Effective Product CultureBoSON22 | Janna Bastow | From Vanity Metrics to Effective Product Culture
BoSON22 | Janna Bastow | From Vanity Metrics to Effective Product Culture
 
Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018)
Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018)Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018)
Forecasting with Less Effort and More Accuracy (Agile Camp NY 2018)
 
Self Care Green and Pink Newsletter Elegant Minimal Pastel Presentation.pptx
Self Care Green and Pink Newsletter Elegant Minimal Pastel Presentation.pptxSelf Care Green and Pink Newsletter Elegant Minimal Pastel Presentation.pptx
Self Care Green and Pink Newsletter Elegant Minimal Pastel Presentation.pptx
 
How to inspire content and dev teams to get the results in search you need.
How to inspire content and dev teams to get the results in search you need.How to inspire content and dev teams to get the results in search you need.
How to inspire content and dev teams to get the results in search you need.
 
SearchLove San Diego 2019 - Vince Nero - Creating Great Content in Boring Ind...
SearchLove San Diego 2019 - Vince Nero - Creating Great Content in Boring Ind...SearchLove San Diego 2019 - Vince Nero - Creating Great Content in Boring Ind...
SearchLove San Diego 2019 - Vince Nero - Creating Great Content in Boring Ind...
 
Holiday catalog testing with Christopher Werler
Holiday catalog testing with Christopher WerlerHoliday catalog testing with Christopher Werler
Holiday catalog testing with Christopher Werler
 

Último

Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introductionsanjaymuralee1
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionajayrajaganeshkayala
 
AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)Data & Analytics Magazin
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxDwiAyuSitiHartinah
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Vladislav Solodkiy
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best PracticesDataArchiva
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationGiorgio Carbone
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024Becky Burwell
 
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Guido X Jansen
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...PrithaVashisht1
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptaigil2
 
YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.JasonViviers2
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxVenkatasubramani13
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructuresonikadigital1
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?sonikadigital1
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerPavel Šabatka
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityAggregage
 

Último (17)

Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introduction
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual intervention
 
AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - Presentation
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024
 
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .ppt
 
YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptx
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructure
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?
 
The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayer
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
 

Giulia Panozzo | BrightonSEO Measurefest | October 2022

  • 1. #brightonseo Winner or Loser? Analysing Your Test Results with Causal Impact on R Studio Slideshare.Net/GiuliaPanozzo1 @SequinsNSearch Giulia Panozzo
  • 2. Giulia Panozzo - @SequinsNSearch #brightonseo Pre- implementation data Post- implementation data
  • 3. Giulia Panozzo - @SequinsNSearch #brightonseo
  • 4. Giulia Panozzo - @SequinsNSearch #brightonseo Why do we care about Causal Impact?
  • 5. Giulia Panozzo - @SequinsNSearch #brightonseo Because Testing is Hard!
  • 6. Giulia Panozzo - @SequinsNSearch #brightonseo
  • 7. Giulia Panozzo - @SequinsNSearch #brightonseo
  • 8. Giulia Panozzo - @SequinsNSearch #brightonseo
  • 9. Giulia Panozzo - @SequinsNSearch #brightonseo Example ‘Let’s add a price point to the title tag!’
  • 10. Giulia Panozzo - @SequinsNSearch #brightonseo Example ‘Let’s add a price point to the title tag!’
  • 11. Giulia Panozzo - @SequinsNSearch #brightonseo ‘But will this help bring more traffic in?’
  • 12. Giulia Panozzo - @SequinsNSearch #brightonseo ‘Well…’
  • 13. Giulia Panozzo - @SequinsNSearch #brightonseo
  • 14. Giulia Panozzo - @SequinsNSearch #brightonseo However, with Causal Impact… X
  • 15. Giulia Panozzo - @SequinsNSearch #brightonseo Causal Impact gives you the confidence to leverage statistically significant results and drive changes at scale
  • 16. Giulia Panozzo - @SequinsNSearch #brightonseo ‘But will this help bring more traffic in?’
  • 17. Giulia Panozzo - @SequinsNSearch #brightonseo YES (Most likely) NO (Most likely)
  • 18. Giulia Panozzo - @SequinsNSearch #brightonseo You can use Causal Impact on a number of domains, not only on SEO tests!
  • 19. Giulia Panozzo - @SequinsNSearch #brightonseo Impact of feature changes on app installs
  • 20. Giulia Panozzo - @SequinsNSearch #brightonseo Impact of influencer campaigns
  • 21. Giulia Panozzo - @SequinsNSearch #brightonseo Impact of offline events and campaigns
  • 22. Giulia Panozzo - @SequinsNSearch #brightonseo And almost any time series data
  • 23. Giulia Panozzo - @SequinsNSearch #brightonseo If you run Causal Impact on R Studio, it’s free and open-source
  • 24. Giulia Panozzo - @SequinsNSearch #brightonseo
  • 25. Giulia Panozzo - @SequinsNSearch #brightonseo What is Causal Impact And how it can help your strategy
  • 26. Giulia Panozzo - @SequinsNSearch #brightonseo What is Causal Impact?
  • 27. Giulia Panozzo - @SequinsNSearch #brightonseo Powerful package to analyse data and infer the cumulative impact of a change in a time series
  • 28. Giulia Panozzo - @SequinsNSearch #brightonseo Based on BSTS (Bayesian Structural Time Series) statistical model
  • 29. Giulia Panozzo - @SequinsNSearch #brightonseo Uses past data to predict the outcome in the absence of the treatment (the counterfactual)
  • 30. Giulia Panozzo - @SequinsNSearch #brightonseo Defines the impact by measuring the deviation of the actual VS predicted outcome
  • 31. Giulia Panozzo - @SequinsNSearch #brightonseo What is Causal Impact?
  • 32. Giulia Panozzo - @SequinsNSearch #brightonseo How can it help us in marketing?
  • 33. Giulia Panozzo - @SequinsNSearch #brightonseo How can it help us in marketing? Example of a clear winner from a title tag change Clicks: +58% CTR: +38% Position: -15%
  • 34. Giulia Panozzo - @SequinsNSearch #brightonseo It can validate proposed strategy changes in case of any doubts
  • 35. Giulia Panozzo - @SequinsNSearch #brightonseo It’s great to clearly show stakeholders the impact of our team’s work
  • 36. Giulia Panozzo - @SequinsNSearch #brightonseo It can help forecast the direction of changes at scale and help make a case for more resources
  • 37. Giulia Panozzo - @SequinsNSearch #brightonseo How can it help us in marketing? Example of a clear loser from a title tag change
  • 38. Giulia Panozzo - @SequinsNSearch #brightonseo By clearly identifying a winner or loser, we can understand what works and doesn’t work for our audience
  • 39. Giulia Panozzo - @SequinsNSearch #brightonseo How to run your analysis with Causal Impact on R Studio
  • 40. Giulia Panozzo - @SequinsNSearch #brightonseo 1. Download R Studio Download R first https://cran.r-project.org/ Download RStudio https://www.rstudio.com/products/rstudio/download/
  • 41. Giulia Panozzo - @SequinsNSearch #brightonseo 1. Download R Studio
  • 42. Giulia Panozzo - @SequinsNSearch #brightonseo 1.1 Install Causal Impact
  • 43. Giulia Panozzo - @SequinsNSearch #brightonseo 2. Prepare the data
  • 44. Giulia Panozzo - @SequinsNSearch #brightonseo 2. Prepare the data
  • 45. Giulia Panozzo - @SequinsNSearch #brightonseo The first column is always your test group. Other columns can be used as control groups if they are a good fit
  • 46. Giulia Panozzo - @SequinsNSearch #brightonseo The pre-period should be at least twice as long as the post-period, to allow the model to plot the actual VS predicted outcome
  • 47. Giulia Panozzo - @SequinsNSearch #brightonseo Any column with 0 should be either removed or corrected
  • 48. Giulia Panozzo - @SequinsNSearch #brightonseo Isolated 0 in data set Multiple 0s VS
  • 49. Giulia Panozzo - @SequinsNSearch #brightonseo 3. Run the script!
  • 50. Giulia Panozzo - @SequinsNSearch #brightonseo Choose file to import
  • 51. Giulia Panozzo - @SequinsNSearch #brightonseo Check preview
  • 52. Giulia Panozzo - @SequinsNSearch #brightonseo Set pre and post periods
  • 53. Giulia Panozzo - @SequinsNSearch #brightonseo It’s a winner!
  • 54. Giulia Panozzo - @SequinsNSearch #brightonseo
  • 55. Giulia Panozzo - @SequinsNSearch #brightonseo
  • 56. Giulia Panozzo - @SequinsNSearch #brightonseo Now give it a go! Request access to this script here
  • 57. Giulia Panozzo - @SequinsNSearch #brightonseo What I’ve learned from (several) trials and errors…
  • 58. Giulia Panozzo - @SequinsNSearch #brightonseo The date column should always be removed when using this script
  • 59. Giulia Panozzo - @SequinsNSearch #brightonseo Column titles can error out if they contain special characters, spaces, capitalised letters
  • 60. Giulia Panozzo - @SequinsNSearch #brightonseo Start small, then expand your datasets with additional controls and features once you’re comfortable with the script
  • 61. Giulia Panozzo - @SequinsNSearch #brightonseo Statistics VS Everything Else Limitations & Takeaways
  • 62. Giulia Panozzo - @SequinsNSearch #brightonseo 1. External events can impact your data
  • 63. Giulia Panozzo - @SequinsNSearch #brightonseo Google algo updates
  • 64. Giulia Panozzo - @SequinsNSearch #brightonseo Tools tracking failures
  • 65. Giulia Panozzo - @SequinsNSearch #brightonseo Engineering releases
  • 66. Giulia Panozzo - @SequinsNSearch #brightonseo Create a document to map internal changes & external events This will allow you to take into account any other known factors and isolate the treatment in the analysis
  • 67. Giulia Panozzo - @SequinsNSearch #brightonseo 2. Mind the Outliers!
  • 68. Giulia Panozzo - @SequinsNSearch #brightonseo
  • 69. Giulia Panozzo - @SequinsNSearch #brightonseo Outliers can originate from…
  • 70. Giulia Panozzo - @SequinsNSearch #brightonseo New product launches (within the test group)
  • 71. Giulia Panozzo - @SequinsNSearch #brightonseo Holidays and seasonal events
  • 72. Giulia Panozzo - @SequinsNSearch #brightonseo Results only from one page Page 1 Page 2 Page 3 Page 4 Page 5
  • 73. Giulia Panozzo - @SequinsNSearch #brightonseo Tracking bugs
  • 74. Giulia Panozzo - @SequinsNSearch #brightonseo You can spot outliers and rule them out by…
  • 75. Giulia Panozzo - @SequinsNSearch #brightonseo Increasing the size of the test group
  • 76. Giulia Panozzo - @SequinsNSearch #brightonseo Adding control groups
  • 77. Giulia Panozzo - @SequinsNSearch #brightonseo Pre-processing your raw data
  • 78. Giulia Panozzo - @SequinsNSearch #brightonseo
  • 79. Giulia Panozzo - @SequinsNSearch #brightonseo
  • 80. Giulia Panozzo - @SequinsNSearch #brightonseo
  • 81. Giulia Panozzo - @SequinsNSearch #brightonseo 3. Beware of confirmation bias
  • 82. Giulia Panozzo - @SequinsNSearch #brightonseo Sometimes, your test will be inconclusive, or might be a loser even when you thought it’d be an easy winner
  • 83. Giulia Panozzo - @SequinsNSearch #brightonseo
  • 84. Giulia Panozzo - @SequinsNSearch #brightonseo In that case, you can run the test a little longer, or repeat the test with bigger groups
  • 85. Giulia Panozzo - @SequinsNSearch #brightonseo If it’s still inconclusive or a loser, it’s probably best to revert the change and focus on other tests
  • 86. Giulia Panozzo - @SequinsNSearch #brightonseo References and useful resources • How we use causal impact analysis to validate campaign success - Part and Sum • Measuring No-ID Campaigns with Causal Impact - Remerge & Alicia Horsch • Causal Impact – Data Skeptic • R Studio on GitHub • The Comprehensive R Archive Network • Causal Impact for App Store Analysis - William Martin