4. What is Robots.txt?
The robots.txt is a text file webmasters create to
instruct robots (typically search engine robots) on how
to crawl & index pages on their website.
What is Sitemap (XML)?
Sitemap (XML) is a list of pages on your
Website. It is easiest way to give Google, and
other search engines, information about your
site and may speed up Google’s crawlers
finding you.
8. Comparing the number of pages from the sitemap vs. the number of pages
already indexed by Google.
On Webmaster Tools, go to “Optimization”, then to “Sitemap”:
9. For Index Status - Go to “Health”, and then to “Index Status”:
Rough estimation of how
many pages from the site
have been already
indexed by Google.
10. HTML Improvements
Issues that are causing duplication within your site.
Identify pages (URL parameters, session IDs and/or pagination problems).
11. Site’s Crawl Error Data
Check for URLs
Particularly the extended URL strings being crawled by Google.
12. Crawl Errors are issued encountered while crawling your site
Crawl Error Types- Soft 404 Errors
HTTP Errors
Not Found Errors
Sitemap Errors
13. • This feature will take the error out of your list unless it isn’t
resolved and will return again.
Mark as
Fixed
14. Reviving the page on a new or old URL (if the non-
existent page is important), then 301 redirect the old
URL to the new one.
301 redirecting the page to another relevant
page/category (if the page is linked from external
websites)
Removing internal links pointing to the
404 page (if page is not that important)
Fixing the page (if issue was caused by
server-end or coding errors).
15.
16. The HTML Improvements page shows you potential issues Google
found when crawling and indexing your site.
HTML Improvements is another feature of WMT which can be found
under the “Optimization” tab.
17. Duplicate meta descriptions Long title tags Short title tags
Long meta descriptions Duplicate title
Non-informative title tags
tags
Short meta descriptions Missing title tags Non-indexable content
The list for each potential page-level issue can guide you on what
changes/improvements to implement for the pages that search crawlers might have
found to be causing indexation problems for your site.
18.
19. To view Site speed Just go
to “Labs”, and then choose
“Site Performance”.
Gaining insights
Will provide performance
overview.
Tools - Google’s Page
Speed or Pingdom