Presented live at Mozcon 2019:
Google's algorithms have undergone significant changes in recent years. Traditional ranking signals don't hold the same sway they used to, and they're being usurped by factors like UX and brand that are becoming more important than ever before. What's an SEO to do? The answer lies in testing.
Sharing original data and results from clients, Rob will highlight the necessity of testing, learning, and iterating your work, from traditional UX testing to weighing the impact of technical SEO changes, tweaking on-page elements, and changing up content on key pages. Actionable processes and real-world results abound in this thoughtful presentation on why you should be testing SEO changes, how and where to run them, and what kinds of tests you ought to consider for your circumstances.
3. Everything Changes
When you’re newer to an industry / an organization
/ a department, you have the opportunity to ask
questions like “why do we do this thing?” and “why do
we do it this way?” and you don’t put up with the
answer “because that’s the way we’ve always done it..”
5. SEO is Changing
15 years ago, the ranking factors we talked about
were almost exclusively around term-relevance and
the link-based authority of the site & page.
6. SEO is Changing
Is the page
relevant to the
term?
Is the page/site
authoritative?
7. SEO is Changing
The ‘ranking factors’ were determined and tweaked
by a team of humans inside Google.
8. SEO is Changing
In response, we ran correlation studies to try to
reverse engineer what they were doing.
9. Google’s Goal:
Portal to the Internet
Google’s aspiration was to be your jumping-off point
onto the rest of the web. If you clicked through to a
site and had a good internet experience, it reflected
well on them. And if you had a bad experience, it
reflected poorly on them for sending you to that site.
10. Malware & viruses
Paywall / registration-wall on landing page
Out of stock products
Out of date information
Pages that make it hard to find information
Pages that are full of advertising
Slow Website
Bad User Experiences
12. Google Has More and More Data
But Google ended up being able to collect more
data about users and their paths than anyone ever
imagined.
13. Google Has More and More Data
(Though we don’t know which of these data sources
they’ve ever used as direct or indirect inputs.)
14. Google is Comfortable with Machine Learning
https://www.wired.com/2016/06/how-google-is-remaking-itself-as-a-machine-learning-first-company/
They also become much more amenable to using
machine learning to process all of this data. This has
better allowed user satisfaction to be measured
directly. A good example would be if a site ranked
number one, but everyone who clicked on it then
clicked back to the results and selected something
else, then maybe it shouldn’t be ranking there.
17. Contemporary SEO
Page 1 Page 2+
For any search, most of the clicks are on page 1,
rather than page 2 and beyond.
18. Contemporary SEO
Page 1 Page 2+
Very little
user data
Loads of
user data
So there’s a lot more user data available to Google
about the engagement with page 1. And there’s very
little data about the deeper results.
19. Contemporary SEO
Page 1 Page 2+
Some
correlation
with links
Data via: distilled.net
On pages 2+, where there’s not much user data, we
see a pretty reasonable correlation between links
and rankings.
20. Contemporary SEO
Page 1 Page 2+
Some
correlation
with links
Less
correlation
with links
Data via: distilled.net
But at the bottom of page 1, the correlation is
weaker.
21. Contemporary SEO
Page 1 Page 2+
Some
correlation
with links
Weakest
correlation
with links
Data via: distilled.net
And at the top of the first page, there’s barely any
correlation between rankings & links!
Less
correlation
with links
22. Contemporary SEO
Page 1
And…
this correlation is getting
weaker every year.
Data via: distilled.net
Less
correlation
with links
Weakest
correlation
with links
23. Contemporary SEO
SearchVolume
Very little
user data
Loads of
user data
Data via: distilled.net
When we think about the head & long tail of
keywords, we know that there is much more user
data available for head terms than then long tail.
24. Contemporary SEO
SearchVolume
Rankings more based on
user engagement metrics
Rankings are
more link-based
Data via: distilled.net
And again, we see that the long tail rankings are
more link based. High volume head terms are more
dictated by user engagement, brand strength, etc.
25. Contemporary SEO
Larry Kim’s research into click-through-rate’s influence on rankings:
moz.com/blog/higher-organic-click-through-conversion-rates-rankings
26. Contemporary SEO
Brian Dean’s conclusions about the impact of bounce rate on rankings:
backlinko.com/search-engine-ranking
28. Contemporary SEO
Google have even been somewhat open about the
fact that they are at least looking at these kinds of
metrics.
29. Contemporary SEO
“The team ran through various data points, like
what percent of users clicked through a
picture-link and then quickly clicked back (a
bad sign), or whether there was a significant
increase in the time until they made their first
interaction with the results (also bad).”
https://www.cnbc.com/2018/09/17/google-tests-changes-to-its-search-algorithm-how-search-works.html
30. User Metrics as Ranking Factors
When we see a site like this, with pop-ups and ads
obscuring the content, we would expect to see
search visibility declining.
31. User Metrics as Ranking Factors
Which is exactly what’s happened.
32. User Metrics as Ranking Factors
This site added a registration form before you could
see any of the content.
33. User Metrics as Ranking Factors
And they have been losing search visibility.
41. @RobOusbey
I would imagine that people are clicking back away
from Forbes, to get back to the search results. Or
possibly even avoiding clicking on a Forbes result in
the first place.
42. SEO is Changing
700
new linking sites,
every day
And even though they’re still earning a lot of links….
53. The SEO Process
+2% +2% +2% +2% +2%
Outcome:
14% traffic increase
+2% +2%
But do you know if each change contributed an
equal amount?
54. The SEO Process
0% 0% 0% 0% 0%
Outcome:
14% traffic increase
+8% +6%
Or did only some of the changes have an impact?
55. The SEO Process
0% 0% -1% -3% -4%
Outcome:
14% traffic increase
+12% +10%
Or – worse still – what if some of the changes
actually harmed traffic? (But the success of the other
changes masked this effect.)
56. The SEO Process
0% 0% -1% -3% -4%
Outcome:
22% traffic increase
+12% +10%
If you’d known what the impact of each change was going to be, you would have only applied the successful ones! You’d have
saved time & effort in implementation, and had MORE traffic!
The SEO process of the future would let us isolate every single change, and measure the impact of them separately.
57. Testing the Impact of SEO Changes
Before
500k visits
/month
How do we test the impact on any one change to a
website? Here’s how you might go about it…
59. Testing the Impact of SEO Changes
Before
500k visits
/month
After
600k visits
/month
You could look at how much traffic you had before
and after making the change to your site.
61. Testing the Impact of SEO Changes
If your site is very seasonal, this won’t work.
62. Testing the Impact of SEO Changes
If your industry has a general trend, it won’t work.
63. Testing the Impact of SEO Changes
If there are any UX or general SERP changes during
the period of your test, it will throw off your data.
64. We need to test against a parallel universe
You really need to be able to compare two parallel
universes: one where we made the change, and one
where we didn’t do anything.
66. Testing the Impact of SEO Changes
Animals
Dogs UnicornsCats Badgers
Let’s use this as the section of a demo site.
67. Testing the Impact of SEO Changes
H1 Badgers
Old Design
Intro Copy
Image
68. Badgers
Testing the Impact of SEO Changes
H1 Badgers
Old Design
Intro Copy
Image
H1
Proposed Design
Video
Bullet List
69. Testing the Impact of SEO Changes
Animals
Dogs UnicornsCats Badgers
We’ll keep half the site as it is at the moment…
70. Testing the Impact of SEO Changes
Animals
Cats Dogs Unicorns Badgers
… and apply our proposed changes to the other half
of the site.
71. Testing the Impact of SEO Changes
Control group
Variant group
1000 -
5000 -
4000 -
3000 -
2000 -
6000 -
7000 -
Before the change, traffic to each group will be able
the same.
72. Testing the Impact of SEO Changes
Control group
Variant group
1000 -
5000 -
4000 -
3000 -
2000 -
6000 -
7000 -
Test start date
Then we roll those changes out to the variant group.
73. Testing the Impact of SEO Changes
Control group
Variant group
1000 -
5000 -
4000 -
3000 -
2000 -
6000 -
7000 -
Test start date
Then we wait for Google to crawl/index the updated
pages.
74. Testing the Impact of SEO Changes
Control group
Variant group
Test start date
1000 -
5000 -
4000 -
3000 -
2000 -
6000 -
7000 -
We’re not looking for the change in traffic to the variant group,
but to see if there’s a gap between the variant and control group.
75. Testing the Impact of SEO Changes
1000 -
5000 -
4000 -
3000 -
2000 -
6000 -
7000 -
In this case, traffic to the variant group moved above
the control group, so we know the change was an
effective one.
76. Testing the Impact of SEO Changes
Positive Test
Roll out to
whole site
There are three potential outcomes from a test.
77. Testing the Impact of SEO Changes
Positive Test
Roll out to
whole site
Revert the
changes
Negative Test
78. Testing the Impact of SEO Changes
Positive Test
Roll out to
whole site
Null Test
Make an
informed decision
Revert the
changes
Negative Test
79. Testing the Impact of SEO Changes
Split pages
into equal groups
Apply changes to
variant group
Create predictive
model for traffic to
variant group
Analyze impact of
the changes
This is the process we are running.
80. Distilled has developed a tool to do all of this for you.
We call it our “Optimization Delivery Network”:
DistilledODN.com
83. I’m going to share some examples from clients who
are under NDA, but I’m going to use BestBuy.com as
the site to take screenshots from for these demos.
96. Control:
Category pages still have ‘SEO content’
Variant:
‘SEO content’ removed from these pages
Category Page SEO Content Test
97. Control:
Category pages still have ‘SEO content’
Variant:
‘SEO content’ removed from these pages
Result:
no change in traffic
Category Page SEO Content Test
98. Control:
Category pages still have ‘SEO content’
Variant:
‘SEO content’ removed from these pages
Result:
3.1% increase in traffic
Category Page SEO Content Test
99. Control:
Category pages still have ‘SEO content’
Variant:
‘SEO content’ removed from these pages
Result:
3.8% decrease in traffic
Category Page SEO Content Test
115. Title Tag / Meta Description Tests
Before:
After:
116. Control:
Title tags remain focusses on main topic
Variant:
Title tag mentions other topics covered on page
Result:
10% decrease in traffic
Title Tag / Meta Description Tests
117. Title Tag / Meta Description Tests
6%: Positive Tests
37%: Null Tests 57%: Negative Tests
129. MAY 2014
“our indexing system has been rendering a
substantial number of web pages more like an
average user’s browser with JavaScript turned on."
OCT 2015
“We are generally able to render and understand
your web pages like modern browsers.”
Javascript Visibility Test
141. Control:
Pages had honest dateModified dates
Variant:
Pages had dateModified set to today’s date
Fresh Content Test
142. Control:
Pages had honest dateModified dates
Variant:
Pages had dateModified set to today’s date
Result:
8% increase in traffic
Fresh Content Test
154. DISTILLED.NET
SEE ALL OUR FREE VIDEOS
AND BLOG POSTS:
CHECK OUT OUR
SEO A/B TESTING PLATFORM:
DISTILLEDODN.COM
@RobOusbey
155. Photo Credits
Film Rolls: Colby Stopa on Visual hunt / CC BY
Face Icons: Icon made by Freepik from www.flaticon.com
Google and the Google logo are registered trademarks of Google LLC, used with permission.
@RobOusbey @Distilled