Find out how you can use Python to analyse the language of the SERPs for valuable insights on what your customers want and how this can be applied to improve the performance of your SEO campaign.

1. Mining the SERPs for SEO,
Content & Customer
Insights
Rory Truesdale // Conductor
http://cndr.co/brighton
@RoryT11

2. About Me
Rory Truesdale
•SEO Strategist at Conductor
•EMEA SEO lead for WeWork
Get In Touch
brightonseo@conductor.com
@RoryT11
@RoryT11

3. Get The Slides
@RoryT11
http://cndr.co/brighton

4. •SERPs are a great resource to learn what Google
‘thinks’ our customers want
•Workflows that will help you understand the intent of
the people you want to reach
•How to use these insights to improve the quality of your
on-page optimisation
What To Expect
@RoryT1

5. That’s how often Google rewrites the SERP
displayed meta description

6. WHY?

7. To make
SEOs sad?

8. Just for a
laugh?

9. Nope…

10. It’s because Google thinks
it is smarter than us

11. Intriguing…
Can we use that to our advantage?

12. Yes, we
can!
(sorry, that was
the last puppy pic)

13. How?
@RoryT11

14. We can deconstruct &
analyse the language in
SERP displayed content
to learn what Google
thinks our customers
are interested in
@RoryT1

15. Curious?
This is important
because we are
in the age of
semantic search
@RoryT1

16. Google isn’t ranking a page based on how
it uses a keyword.
Google provides accurate results based on
intent, query context & word relationships.
On-page Optimisation
@RoryT1

17. • User intent
• Query context
•Topical relevance
• Word relationships
Target the keyword, but optimise for this.
On-page Optimisation
@RoryT1

18. Understand
customer intent
& desire to better
tailor your
messaging
@RoryT1

19. Structure landing
pages to help
Google understand
context & how it
meets the needs
of the searcher
@RoryT1

20. Build more
meaningful
online
experiences that
better convert
website visitors
@RoryT1

21. Your
Toolkit
@RoryT11

22. You need
SERP content
There are
three ways
you can get
this.
@RoryT1

23. Scrape at scale with
Screaming Frog
Follow these instructions
@RoryT1
Option
A

24. Option
B
Get SERP content via an
API

25. Option
C
Get SERP content using the
Scraper Chrome extension
Get
Scraper

26. There are four
ways you can
get this.
You need
Jupyter Notebook
What is
that?

27. The Jupyter Notebook is an open-source web
application that allows you to create and share
documents that contain live code, equations,
visualizations and narrative text. Uses include: data
cleaning and transformation, numerical simulation,
statistical modeling, data visualization, machine
learning, and much more.
Jupyter.org
@RoryT1

28. Stumped?
Me too…
Here’s my definition

29. Jupyter Notebook is an environment on my laptop
where I can learn Python by copying scripts created
by people significantly smarter than me and
breaking them or making them do something
slightly different.
RoryTruesdale
Python Charlatan
@RoryT1

30. Resources to get started…
Jupyter
Notebook –
Getting
Started Guide
Robin Lord
Find
scripts
Paul Shapiro JR Oakes Hamlet
Batista
Find
scripts
Find
scripts

31. You’ll end up
with…
@RoryT11

32. Your SERP content in a CSV@RoryT1

33. Imported into Jupyter Notebook
@RoryT1

34. You’re
ready to
use Python
to analyse
the SERPs!
@RoryT1

35. There’s a
treat for you.

36. I’ll share a link to a Dropbox
with everything you need to get
you started
@RoryT11

37. Before we
dive in…

38. Start by
cleaning
your SERP
content
@RoryT1

39. Lower case avoids
duplication &
punctuation adds
no value to this
analysis
Lower Case &
Remove
Punctuation
@RoryT1

40. Stop words are
commonly
occurring words
that don’t improve
our analysis
Remove
Stop
Words
@RoryT1

41. The process of
chopping up a
sentence into
individual pieces,
called ‘tokens’
Tokenization
@RoryT1

42. The process of
converting a word
to its root (i.e.
“playing” becomes
“play”)
Lemmatization
(optional)
@RoryT1

43. @RoryT11

44. @RoryT11

45. How many times
does a word or
combination of
words appear in
your SERP
content?
Co-occurrence
@RoryT1

46. Co-
occurrence
Snapshot of
phrases
frequently
occurring in
the SERPs
@RoryT1

47. Co-
occurrence
Demonstrates
the topics
competitors
cover on landing
pages
@RoryT1

48. Co-
occurrence
Understand the
types of phrases
that Google sees
as semantically
relevant to a target
keyword set

49. •Additional source of data for keyword research
•Identify topical content gaps on landing pages
•Optimise landing page content by incorporating
semantically relevant phrases
HOW CAN WE APPLY THIS?
@RoryT1

50. Cost:
Range:
Time to Charge:
Battery Size/Capacity:
All Wheel Drive:
Towing Capacity:
Semi-Conductor SERP XLT: Product Page
£
44,360 MSV
9,620 MSV
7,470 MSV
380 MSV
3,040 MSV
180 MSV

51. What are the most
frequently
occurring nouns,
verbs &
adjectives in a
SERP?
Part of Speech
Tagging
@RoryT1

52. PoS
Tagging
Uncover the
phrases or topics
you should include
in your landing
pages to rank for a
term
Nouns (people, place, thing)
@RoryT1

53. PoS
Tagging
Get clues around
how Google is
interpreting the
context and intent
of a search
Verbs (action or state)
@RoryT1

54. PoS
Tagging
Understand the
language and
tone that might
resonate with a
searcher
Adjectives (descriptive word)
@RoryT1

55. PoS
Tagging
Credit Card Example – P1 Verbs
Intent Clues: What is the specific motivation
our searcher has?

56. PoS
Tagging
Credit Card Example - P1 Nouns
Context Clues: Words that clarify meaning &
help us understand what a searcher wants
@RoryT1

57. PoS
Tagging
Credit Card Example - P1 Adjectives
Context Clues: Words that clarify meaning &
help us understand what a searcher wants
@RoryT1

58. •Create landing pages that are aligned with the
intent of a searcher
•Help copywriters understand the language and
desires of a target audience
•Tactically incorporate more semantically relevant
phrases into landing pages
HOW CAN WE APPLY THIS?
@RoryT1

59. Can we use NLP
to uncover topical
trends in the
SERPs to help us
with content
ideation?
Topic
Modelling
@RoryT1

60. Topic
Modelling
Topic modelling is an NLP method that assumes a
corpus contains a mixture of topics. It looks at how
words and phrases co-occur in a corpus and attempts
to group them in coherent themes or topics.
@RoryT1

61. Topic
Modelling
OK, computer. Here’s some words. Group them.
@RoryT1
RoryTruesdale
Cheapening machine
learning since 2019

62. Topic
Modelling
Each bubble
represents a
topic
@RoryT1

63. Topic
Modelling
The bigger
the bubble
the more
prominent
the topic
@RoryT1

64. Topic
Modelling
The further
away the
bubbles are,
the more
distinct those
topic are

65. Topic
Modelling
Get a
breakdown of
the terms our
topics consist
of
@RoryT1

66. Topic
Modelling
The output is an
interactive visual
on topical trends
that can be easily
shared with other
teams
@RoryT1

67. Topic
Modelling
Use Google’s
algorithm to help
us identify areas
of interest for our
audience

68. Topic
Modelling
Uncover topical
trends hidden in
the language of
the SERPs that
can inform
content ideation
@RoryT1

69. •Valuable data point to reference for content
ideation
•Inform internal linking and content
recommendations across a website
•Incorporate topically relevant phrases into existing
pages to improve semantic relevance
HOW CAN WE APPLY THIS?
@RoryT1

70. How can we make
our scripts work
across other data
sources to
understand our
customers?
Other
Useful
Applications
@RoryT1

71. Product
Reviews
@RoryT11

72. Product Reviews
@RoryT11

73. GMB
Reviews
@RoryT11

74. GMB Reviews
@RoryT11

75. Reddit
@RoryT11

76. Reddit
@RoryT11

77. YouTube
Captions
@RoryT11

78. YouTube Captions
@RoryT11

79. Competitors & Top
Ranking Pages
@RoryT11

80. Competitors & Top Ranking Pages
@RoryT11

81. With some minor
tweaks we can
make our scripts
work across a huge
corpus of user-
centric content
Pretty cool, right?
@RoryT1

82. Potential to ramp up and apply sentiment analysis
to these sources for useful visualisations
@RoryT11

83. Deconstruct product reviews to find out what really
matters to customers
•Simple
•Easy to use
•Intuitive
•Buggy
•Slow
@RoryT1

84. A lot to
take
in…what
does it all
mean?
@RoryT11

85. SERPs give us
amazing insight
into what
customers want
@RoryT1

86. Python makes
getting these
insights at scale
accessible
@RoryT1

87. Use these insights
to align landing
pages with intent
and semantic
relevance
@RoryT1

88. Scripts we create
allow us to get these
insights from lots of
other user-centric
sources beyond the
SERPs
@RoryT1

89. http://cndr.co/jupyter
Python Dropbox Link
@RoryT1

90. Get The Slides
@RoryT11
http://cndr.co/brighton

91. • https://www.searchenginejournal.com/scrape-google-serp-custom-extractions/267211/
• https://www.searchenginejournal.com/mine-serps-seo-content-customer-insights/311137/
• https://www.seerinteractive.com/blog/user-testing-serps-an-audience-first-approach-to-seo/
• https://www.dropbox.com/sh/vl5miyt6sgbvmkl/AAC5365YcWTun_EzkQLtixe1a?dl=0 (Jupyter
Notebook tutorial)
• http://www.blindfiveyearold.com/algorithm-analysis-in-the-age-of-embeddings
• https://www.searchenginejournal.com/semantic-search-seo/264037/#close
• https://www.slideshare.net/DawnFitton/natural-language-processing-and-search-intent-
understanding-c3-conductor-2019-dawn-anderson
• https://moz.com/blog/what-is-semantic-search
• https://www.slideshare.net/paulshapiro/redefining-technical-seo-mozcon-2019-by-paul-shapiro
Useful Resources
@RoryT1

92. Thanks
For
Listening!
Conductor.com
@RoryT11
brightonseo@conductor.com