Gareth Beck at Bristol SEO Meetup 26 March 2019

1. Evaluating URLs at scale
26th May 2019

2. What was I thinking?
Maybe, just maybe, should have been called:
A guide to an SEO toolkit

3. Quick BIO
• Head of Digital Marketing at Bray Leino CX – 14 whole days
• Head of SEO - 2 years
• Senior SEO Manager – 2 years
• Ashridge Trees – Marketing Manager – 1 year (was a bad idea)
• Head of SEO – 3 years – SIFT / PracticeWEB
• Run a team of 4 (we’re looking for a technical SEO exec to join us) covering Digital Marketing

4. How many URLs can you check?
10, 100, 1000, 10,0000, a million, 100 million?
There comes a point where you want to look at them without
having to look at individual URLs.

5. URL Tool Kit*
* There are other tools. Many. Many other tools.
Crawling Backlinks Site Speed
SEMRush SEMRush Sitespeed.io
SiteBulb Majestic Lighthouse
Xenu Link Sleuth Ahrefs URL Profiler
Screaming Frog
URLProfiler
Custom Built
Botify
Bright Local
DeepCrawl

6. • Backlink analysis
• Migrations
• Site Structure (Internal linking visualisation)
• Specific Data (authorship?) / Regex? Canonicals
• Angular sites (SPA)
• Content Audits
What context?

7. Backlink analysis

8. Initial Audit
Backlink Analysis
• Get links from Search Console,
Majestic, AHREFS, SEMRUSH, MOZ
• Use list mode in Screaming Frog
• Crawl the links
• See which ones 404 or have long
redirect chains
• Fix them!

9. Initial Audit
Backlink Analysis
• Get links from Search Console,
Majestic, AHREFS, SEMRUSH, MOZ
• Use list mode in Screaming Frog
• Crawl the links
• See which ones 404 or have long
redirect chains
• Fix them!

10. Migration

11. Moving site
Migration
• Crawl the existing site in spider mode
• Crawl the development site in spider mode
• Export HTML pages from existing site
• Change domain name in excel (R-Studio if there are too many)
• Use List mode in SF
• Look at 404’s (these are pages that haven’t been moved to the new site)
• Set-up redirect mapping
• Test (recrawl)

12. Site Structure

13. Site Structure

14. Site Structure
All nodes with no-
weight have been
removed.
L. New int. links: 312
R. Old int. links 374

15. Specific Data

16. Specific Data
• Author Data
• Tag data
• Unencoded URLs
• Anything not captured by default

17. Specific Data
Configuration > Custom > Extraction

18. Specific Data
Right click on the element you want to
Select

19. Specific Data
Paste in to the extractor and rename
This data will be available at the URL level for any page that has it

20. JS and SPA’s

21. Problems
Angular JS and SPA’s

22. Problems
Angular JS and SPA’s

23. Problems Google is pretty good at crawling JS but
not all links on a page will be crawled
(Google I/O May 2018)
It is possible that Google can render the
links in a separate action after the initial
crawl
Other search engines are not as good
and are more likely to have problems.
Angular JS and SPA’s

24. Content Audits

25. • Readability scores
• Sentiment
• Broadcasted keywords
• Page Speed (Desktop vs Mobile)
Content Audits

26. Content Audits

27. Tool Overview

28. Tools
• Three modes
• Spider
• List
• SERP
• Very configurable. Data focused
• Speed of crawl – (can be a DDOS attack, true story.)
• Can specify CDN’s
• Crawler identifying name
• Add GA and SC
• Storage (RAM vs Hard Disk)
• £149 a year

29. Tools
• Report, not crawl
focused
• Hints are great
• Runs and reports on
multiple tests
• Keeps all the audits in a
central place
• £27.50 a month

30. URL Profiler
Tools
• Swiss army knife of tools
• Great at bringing
together metrics in one
place per URL
• £24 a month (2 devices)

31. R Studio
• Steep learning curve
• Can handle large datasets
• Can use machine learning
• Lots of online tutorials
• Keyword mapping (comp analysis)
• Link visualization
• Keyword clustering
• Free

32. Any Questions?