SlideShare una empresa de Scribd logo
1 de 25
Robots.txt & Multi-Site
DNN Instances
A Primer on SEO & Robots.txt in DNN 9.8+
Thank You to Our Sponsors!
Will Strohl
• Using DNN since 2003
• CEO & Founder, Upendo Ventures
• Former DNN Corp Employee
• Former CEO, Hotcakes Commerce (2013-2017)
• DNN MVP Since 2013
• Leader of the Awareness Group
• DNN Author & Technical Editor
• Founder of Day of DNN & DNNCon
• Maintainer of 40+ Open-Source Projects on GitHub
@WillStrohl
WillStrohl.com
Upendo Ventures
• DNN Platform & Evoq
• E-Commerce
• Implementations
• Integrations
• Custom Modules
• Upgrades & Edition Migrations
• Custom Themes
• Training
• Fixing Things Other People Did ;)
@UpendoVentures
UpendoVentures.com
Robots.txt & Multi-Site
DNN Instances
A Primer on SEO & Robots.txt in DNN 9.8+
Objectives
I Hope You Learn…
• The Basics of SEO
• How to Manage Robots.txt
• The Do’s & Don’ts of Robots.txt
• How to Have More than One
Robots.txt in Multi-Portal
Environments
Prerequisites
• You Can Spell SEO ;)
• General Knowledge of How SEO
Works
• General Understanding of Your
Website Implementation
• General Understanding of How to
Administrate DNN
What is SEO?
Search Engine Optimization (“SEO”)
The process of maximizing the number of visitors to a particular
website by ensuring that the site appears high on the list of
results returned by a search engine.
Common SEO Techniques
• Digital Marketing Strategy
• Content Creation
• Regular Content Updates
• Visitor Engagement
• Offsite Content & Links
• Multiple Mediums
• Multiple Channels
• Use HTTPS for All Requests
• Generate a Video Playlist
• Reduce 404 Errors
• Decrease Page Load Times
• Keyword Research
• Keyword Acquisition
• Refine Targeted Content
• Use Schema Markup
• Properly Implement Semantic
Markup
• Optimize for Voice Search
Common SEO Techniques
Keep the Search Engines & Crawlers Informed
Page Meta Data
<meta name="robots" content="noindex" />
<meta name="robots" content="nofollow" />
<meta name="robots" content="noindex,nofollow" />
Default: Index, Follow
Page Meta Data
<meta id="MetaRobots" name="robots" content="INDEX, FOLLOW">
<meta id="MetaRobots" name="robots" content="NOINDEX, NOFOLLOW">
What is Robots.txt?
The robots exclusion standard, also known as the robots
exclusion protocol or simply robots.txt, is a standard used by
websites to communicate with web crawlers and other web
robots. The standard specifies how to inform the web robot
about which areas of the website should not be processed or
scanned.
User-agent
User-agent: [bot identifier/crawler name]
Specifies a bot or crawler to instruct.
User-agent: msnbot
User-agent: Googlebot
User-agent: Slurp
User-agent: *
Allow/Disallow
Allow: [path to crawl]
Disallow: [path to ignore]
Specifies a path where crawlers are or are not allowed to go.
Allow: /
Disallow: /activity-feed/
Crawl-delay
Crawl-delay: [milliseconds]
Indicates a period of time to wait between page requests.
Crawl-delay: 120
Sitemap
Sitemap: [path to sitemap]
Allows you to specify the location of one or more sitemaps for
the search engine to follow.
Sitemap: https://dnncommunity.org/sitemap.xml
Robots.txt Example
Pattern Matching
* is a wildcard pattern, matching any character
$ is the end of the URL
Allow: *.html
Disallow: *.php
Allow: *.aspx$
Robots.txt in DNN
Robots.txt in DNN
• Included since DNN 7.x
• Update for Every Instance
• Difficult to Support Multi-Site
• Used to Require Direct File System Access
• DnnSoftware.com has Good Examples
Multi-Site Robots.txt in DNN
• Requires Rewrite Rules Module
• https://www.iis.net/downloads/microsoft/url-rewrite
• Create a Robots.txt for Each Site
• example.com.robots.txt
• Add a Rewrite Rule for Each Site
• example.com.robots.txt becomes robots.txt
Multi-Site Robots.txt in DNN
Demo
Let’s See This Stuff in Action!
Thank You to Our Sponsors!
Upendo Ventures
Summary
• SEO important on all public sites
• There are many ways to improve SEO (all
are important)
• DNN makes all of these easy
• Robots.txt is fully supported
• Robots.txt files can be edited directly in
the Config Manager
• Be sure to throttle bots using Crawl-
delay
• Multi-site instances require some more
configuration
• Review robots.txt after any major
roll-out
We use technology to help your business
change people’s lives
UpendoVentures.com
solutions@upendoventures.com
(650) 381-9160
548 Market St. #65401
San Francisco, CA 94104

Más contenido relacionado

La actualidad más candente

Version Control, Writers, and Workflows
Version Control, Writers, and WorkflowsVersion Control, Writers, and Workflows
Version Control, Writers, and Workflowsstc-siliconvalley
 
K15t - How Atlassian Does Docs
K15t - How Atlassian Does DocsK15t - How Atlassian Does Docs
K15t - How Atlassian Does DocsK15t
 
Introduce iRedMail Open Source Mail Server Solution
Introduce iRedMail Open Source Mail Server SolutionIntroduce iRedMail Open Source Mail Server Solution
Introduce iRedMail Open Source Mail Server SolutionZhangHuangbin
 
Forcelandia Salesforce CI
Forcelandia Salesforce CIForcelandia Salesforce CI
Forcelandia Salesforce CIDaniel Hoechst
 
WordPress Theme Reviewers Team
WordPress Theme Reviewers TeamWordPress Theme Reviewers Team
WordPress Theme Reviewers TeamMario Peshev
 
OneDrive, TwoDrive, Whiterive, BlueDrive (hahaha)
OneDrive, TwoDrive, Whiterive, BlueDrive (hahaha)OneDrive, TwoDrive, Whiterive, BlueDrive (hahaha)
OneDrive, TwoDrive, Whiterive, BlueDrive (hahaha)SPC Adriatics
 
DocOps: Documentation at the Speed of Agile
DocOps: Documentation at the Speed of AgileDocOps: Documentation at the Speed of Agile
DocOps: Documentation at the Speed of AgileMary Connor
 
Hands-on with Source Control: Using MadCap Flare with a Cloud Source Control ...
Hands-on with Source Control: Using MadCap Flare with a Cloud Source Control ...Hands-on with Source Control: Using MadCap Flare with a Cloud Source Control ...
Hands-on with Source Control: Using MadCap Flare with a Cloud Source Control ...docguy
 
Platforms based on WordPress
Platforms based on WordPressPlatforms based on WordPress
Platforms based on WordPressMario Peshev
 
Building and selling a commercial WordPress plugin
Building and selling a commercial WordPress pluginBuilding and selling a commercial WordPress plugin
Building and selling a commercial WordPress plugindanlester1
 
Getting started with mobile application development
Getting started with mobile application developmentGetting started with mobile application development
Getting started with mobile application developmentColdFusionConference
 
SenchaCon 2016: Being Productive with the New Sencha Fiddle - Mitchell Simoens
SenchaCon 2016: Being Productive with the New Sencha Fiddle - Mitchell Simoens  SenchaCon 2016: Being Productive with the New Sencha Fiddle - Mitchell Simoens
SenchaCon 2016: Being Productive with the New Sencha Fiddle - Mitchell Simoens Sencha
 
O365Con18 - Git and GitHub - Rick van Rousselt
O365Con18 - Git and GitHub - Rick van RousseltO365Con18 - Git and GitHub - Rick van Rousselt
O365Con18 - Git and GitHub - Rick van RousseltNCCOMMS
 
WordPress Security 101 - Meetup Nairobi March 2020
WordPress Security 101 - Meetup Nairobi March 2020 WordPress Security 101 - Meetup Nairobi March 2020
WordPress Security 101 - Meetup Nairobi March 2020 stk_jj
 
Visual Studio Code vs. Sublime Text
Visual Studio Code vs. Sublime TextVisual Studio Code vs. Sublime Text
Visual Studio Code vs. Sublime TextAhmed Bouhuolia
 
Resources for Navigating Drupal Upgrades: Versions 6 Through 8 And What It Me...
Resources for Navigating Drupal Upgrades: Versions 6 Through 8 And What It Me...Resources for Navigating Drupal Upgrades: Versions 6 Through 8 And What It Me...
Resources for Navigating Drupal Upgrades: Versions 6 Through 8 And What It Me...Steve Kessler
 
NSA - No thanks - Build your own cloud with OpenStack and Cloud Foundry | any...
NSA - No thanks - Build your own cloud with OpenStack and Cloud Foundry | any...NSA - No thanks - Build your own cloud with OpenStack and Cloud Foundry | any...
NSA - No thanks - Build your own cloud with OpenStack and Cloud Foundry | any...anynines GmbH
 

La actualidad más candente (20)

Version Control, Writers, and Workflows
Version Control, Writers, and WorkflowsVersion Control, Writers, and Workflows
Version Control, Writers, and Workflows
 
K15t - How Atlassian Does Docs
K15t - How Atlassian Does DocsK15t - How Atlassian Does Docs
K15t - How Atlassian Does Docs
 
Introduce iRedMail Open Source Mail Server Solution
Introduce iRedMail Open Source Mail Server SolutionIntroduce iRedMail Open Source Mail Server Solution
Introduce iRedMail Open Source Mail Server Solution
 
Forcelandia Salesforce CI
Forcelandia Salesforce CIForcelandia Salesforce CI
Forcelandia Salesforce CI
 
WordPress Theme Reviewers Team
WordPress Theme Reviewers TeamWordPress Theme Reviewers Team
WordPress Theme Reviewers Team
 
RavenDB 3.5
RavenDB 3.5RavenDB 3.5
RavenDB 3.5
 
OneDrive, TwoDrive, Whiterive, BlueDrive (hahaha)
OneDrive, TwoDrive, Whiterive, BlueDrive (hahaha)OneDrive, TwoDrive, Whiterive, BlueDrive (hahaha)
OneDrive, TwoDrive, Whiterive, BlueDrive (hahaha)
 
DocOps: Documentation at the Speed of Agile
DocOps: Documentation at the Speed of AgileDocOps: Documentation at the Speed of Agile
DocOps: Documentation at the Speed of Agile
 
Hands-on with Source Control: Using MadCap Flare with a Cloud Source Control ...
Hands-on with Source Control: Using MadCap Flare with a Cloud Source Control ...Hands-on with Source Control: Using MadCap Flare with a Cloud Source Control ...
Hands-on with Source Control: Using MadCap Flare with a Cloud Source Control ...
 
Platforms based on WordPress
Platforms based on WordPressPlatforms based on WordPress
Platforms based on WordPress
 
Building and selling a commercial WordPress plugin
Building and selling a commercial WordPress pluginBuilding and selling a commercial WordPress plugin
Building and selling a commercial WordPress plugin
 
Getting started with mobile application development
Getting started with mobile application developmentGetting started with mobile application development
Getting started with mobile application development
 
SenchaCon 2016: Being Productive with the New Sencha Fiddle - Mitchell Simoens
SenchaCon 2016: Being Productive with the New Sencha Fiddle - Mitchell Simoens  SenchaCon 2016: Being Productive with the New Sencha Fiddle - Mitchell Simoens
SenchaCon 2016: Being Productive with the New Sencha Fiddle - Mitchell Simoens
 
O365Con18 - Git and GitHub - Rick van Rousselt
O365Con18 - Git and GitHub - Rick van RousseltO365Con18 - Git and GitHub - Rick van Rousselt
O365Con18 - Git and GitHub - Rick van Rousselt
 
Maximizing WordPress
Maximizing WordPressMaximizing WordPress
Maximizing WordPress
 
WordPress Security 101 - Meetup Nairobi March 2020
WordPress Security 101 - Meetup Nairobi March 2020 WordPress Security 101 - Meetup Nairobi March 2020
WordPress Security 101 - Meetup Nairobi March 2020
 
Visual Studio Code vs. Sublime Text
Visual Studio Code vs. Sublime TextVisual Studio Code vs. Sublime Text
Visual Studio Code vs. Sublime Text
 
Resources for Navigating Drupal Upgrades: Versions 6 Through 8 And What It Me...
Resources for Navigating Drupal Upgrades: Versions 6 Through 8 And What It Me...Resources for Navigating Drupal Upgrades: Versions 6 Through 8 And What It Me...
Resources for Navigating Drupal Upgrades: Versions 6 Through 8 And What It Me...
 
[Delimon] Unraveling Teams vs Skype for Business
[Delimon] Unraveling Teams vs Skype for Business[Delimon] Unraveling Teams vs Skype for Business
[Delimon] Unraveling Teams vs Skype for Business
 
NSA - No thanks - Build your own cloud with OpenStack and Cloud Foundry | any...
NSA - No thanks - Build your own cloud with OpenStack and Cloud Foundry | any...NSA - No thanks - Build your own cloud with OpenStack and Cloud Foundry | any...
NSA - No thanks - Build your own cloud with OpenStack and Cloud Foundry | any...
 

Similar a DNN Summit: Robots.txt & Multi-Site DNN Instances

Technical SEO - An Introduction to Core Aspects of Technical SEO Best-Practise
Technical SEO - An Introduction to Core Aspects of Technical SEO Best-PractiseTechnical SEO - An Introduction to Core Aspects of Technical SEO Best-Practise
Technical SEO - An Introduction to Core Aspects of Technical SEO Best-PractiseErudite
 
Intro to SharePoint 2013 Branding
Intro to SharePoint 2013 BrandingIntro to SharePoint 2013 Branding
Intro to SharePoint 2013 BrandingThomas Daly
 
Search engine optimization (seo) from Endeca & ATG
Search engine optimization (seo) from Endeca & ATGSearch engine optimization (seo) from Endeca & ATG
Search engine optimization (seo) from Endeca & ATGVignesh sitaraman
 
Search Engine Optimization (Seo) for Developers
Search Engine Optimization (Seo) for DevelopersSearch Engine Optimization (Seo) for Developers
Search Engine Optimization (Seo) for DevelopersMatthew Robinson
 
Guía SEO 2020: Trucos y recomendaciones para desarrolladores y webmasters
Guía SEO 2020: Trucos y recomendaciones para desarrolladores y webmastersGuía SEO 2020: Trucos y recomendaciones para desarrolladores y webmasters
Guía SEO 2020: Trucos y recomendaciones para desarrolladores y webmastersMiguel López Zuleta
 
Seo Beginners Guide SriG Systems
Seo Beginners Guide SriG SystemsSeo Beginners Guide SriG Systems
Seo Beginners Guide SriG SystemsSriG Systems
 
Convert your Full Trust Solutions to the SharePoint Framework (SPFx)
Convert your Full Trust Solutions to the SharePoint Framework (SPFx)Convert your Full Trust Solutions to the SharePoint Framework (SPFx)
Convert your Full Trust Solutions to the SharePoint Framework (SPFx)Brian Culver
 
Code for Startup MVP (Ruby on Rails) Session 1
Code for Startup MVP (Ruby on Rails) Session 1Code for Startup MVP (Ruby on Rails) Session 1
Code for Startup MVP (Ruby on Rails) Session 1Henry S
 
Javascript - Getting started | DevCom ISITCom
Javascript - Getting started | DevCom ISITComJavascript - Getting started | DevCom ISITCom
Javascript - Getting started | DevCom ISITComHamdi Hmidi
 
SIM Digital Marketing COURSES PPT.pptx
SIM Digital Marketing COURSES PPT.pptxSIM Digital Marketing COURSES PPT.pptx
SIM Digital Marketing COURSES PPT.pptxShaileshBone
 
Digital MarketingDigital marketing courses in Pune-SIM.pptx
Digital MarketingDigital marketing courses in Pune-SIM.pptxDigital MarketingDigital marketing courses in Pune-SIM.pptx
Digital MarketingDigital marketing courses in Pune-SIM.pptxMaheshVarade3
 
SIM Digital Marketing COURSE PDF.pdf
SIM Digital Marketing COURSE PDF.pdfSIM Digital Marketing COURSE PDF.pdf
SIM Digital Marketing COURSE PDF.pdfShaileshBone
 
SPS Monaco 2017 - The Lay of the Land of Client-Side Development circa 2017
SPS Monaco 2017 - The Lay of the Land of Client-Side Development circa 2017SPS Monaco 2017 - The Lay of the Land of Client-Side Development circa 2017
SPS Monaco 2017 - The Lay of the Land of Client-Side Development circa 2017Marc D Anderson
 
aOS Canadian Tour Share point migration tips
aOS Canadian Tour Share point migration tipsaOS Canadian Tour Share point migration tips
aOS Canadian Tour Share point migration tipsMike Maadarani
 
13 Things Developers Forget When Launching Public Websites
13 Things Developers Forget When Launching Public Websites13 Things Developers Forget When Launching Public Websites
13 Things Developers Forget When Launching Public WebsitesAJi
 
The Technical SEO Full Course how to do
The Technical SEO  Full Course  how to doThe Technical SEO  Full Course  how to do
The Technical SEO Full Course how to doasadkhan888889990
 

Similar a DNN Summit: Robots.txt & Multi-Site DNN Instances (20)

Technical SEO - An Introduction to Core Aspects of Technical SEO Best-Practise
Technical SEO - An Introduction to Core Aspects of Technical SEO Best-PractiseTechnical SEO - An Introduction to Core Aspects of Technical SEO Best-Practise
Technical SEO - An Introduction to Core Aspects of Technical SEO Best-Practise
 
Intro to SharePoint 2013 Branding
Intro to SharePoint 2013 BrandingIntro to SharePoint 2013 Branding
Intro to SharePoint 2013 Branding
 
Search engine optimization (seo) from Endeca & ATG
Search engine optimization (seo) from Endeca & ATGSearch engine optimization (seo) from Endeca & ATG
Search engine optimization (seo) from Endeca & ATG
 
Search Engine Optimization (Seo) for Developers
Search Engine Optimization (Seo) for DevelopersSearch Engine Optimization (Seo) for Developers
Search Engine Optimization (Seo) for Developers
 
Guía SEO 2020: Trucos y recomendaciones para desarrolladores y webmasters
Guía SEO 2020: Trucos y recomendaciones para desarrolladores y webmastersGuía SEO 2020: Trucos y recomendaciones para desarrolladores y webmasters
Guía SEO 2020: Trucos y recomendaciones para desarrolladores y webmasters
 
web development process WT
web development process WTweb development process WT
web development process WT
 
Wt unit 1 ppts web development process
Wt unit 1 ppts web development processWt unit 1 ppts web development process
Wt unit 1 ppts web development process
 
Digital marketing
Digital marketingDigital marketing
Digital marketing
 
Seo Beginners Guide SriG Systems
Seo Beginners Guide SriG SystemsSeo Beginners Guide SriG Systems
Seo Beginners Guide SriG Systems
 
Convert your Full Trust Solutions to the SharePoint Framework (SPFx)
Convert your Full Trust Solutions to the SharePoint Framework (SPFx)Convert your Full Trust Solutions to the SharePoint Framework (SPFx)
Convert your Full Trust Solutions to the SharePoint Framework (SPFx)
 
Code for Startup MVP (Ruby on Rails) Session 1
Code for Startup MVP (Ruby on Rails) Session 1Code for Startup MVP (Ruby on Rails) Session 1
Code for Startup MVP (Ruby on Rails) Session 1
 
Javascript - Getting started | DevCom ISITCom
Javascript - Getting started | DevCom ISITComJavascript - Getting started | DevCom ISITCom
Javascript - Getting started | DevCom ISITCom
 
SIM Digital Marketing COURSES PPT.pptx
SIM Digital Marketing COURSES PPT.pptxSIM Digital Marketing COURSES PPT.pptx
SIM Digital Marketing COURSES PPT.pptx
 
Digital MarketingDigital marketing courses in Pune-SIM.pptx
Digital MarketingDigital marketing courses in Pune-SIM.pptxDigital MarketingDigital marketing courses in Pune-SIM.pptx
Digital MarketingDigital marketing courses in Pune-SIM.pptx
 
SIM Digital Marketing COURSE PDF.pdf
SIM Digital Marketing COURSE PDF.pdfSIM Digital Marketing COURSE PDF.pdf
SIM Digital Marketing COURSE PDF.pdf
 
SPS Monaco 2017 - The Lay of the Land of Client-Side Development circa 2017
SPS Monaco 2017 - The Lay of the Land of Client-Side Development circa 2017SPS Monaco 2017 - The Lay of the Land of Client-Side Development circa 2017
SPS Monaco 2017 - The Lay of the Land of Client-Side Development circa 2017
 
CAB 2.pptx
CAB 2.pptxCAB 2.pptx
CAB 2.pptx
 
aOS Canadian Tour Share point migration tips
aOS Canadian Tour Share point migration tipsaOS Canadian Tour Share point migration tips
aOS Canadian Tour Share point migration tips
 
13 Things Developers Forget When Launching Public Websites
13 Things Developers Forget When Launching Public Websites13 Things Developers Forget When Launching Public Websites
13 Things Developers Forget When Launching Public Websites
 
The Technical SEO Full Course how to do
The Technical SEO  Full Course  how to doThe Technical SEO  Full Course  how to do
The Technical SEO Full Course how to do
 

Más de Will Strohl

DNN Community Newsletter: An In-Person Review of Recent Open-Source Activity
DNN Community Newsletter: An In-Person Review of Recent Open-Source ActivityDNN Community Newsletter: An In-Person Review of Recent Open-Source Activity
DNN Community Newsletter: An In-Person Review of Recent Open-Source ActivityWill Strohl
 
Unveiling the Secrets of Software Company Transitions: Navigating the Path to...
Unveiling the Secrets of Software Company Transitions: Navigating the Path to...Unveiling the Secrets of Software Company Transitions: Navigating the Path to...
Unveiling the Secrets of Software Company Transitions: Navigating the Path to...Will Strohl
 
DNN CMS Awareness Group Meeting: December 2020
DNN CMS Awareness Group Meeting: December 2020DNN CMS Awareness Group Meeting: December 2020
DNN CMS Awareness Group Meeting: December 2020Will Strohl
 
Tips & Tricks: Working from Home and Staying Productive
Tips & Tricks: Working from Home and Staying ProductiveTips & Tricks: Working from Home and Staying Productive
Tips & Tricks: Working from Home and Staying ProductiveWill Strohl
 
DNN Awareness Meeting July 2019
DNN Awareness Meeting July 2019DNN Awareness Meeting July 2019
DNN Awareness Meeting July 2019Will Strohl
 
DNN-Connect 2019: DNN Horror Stories
DNN-Connect 2019: DNN Horror StoriesDNN-Connect 2019: DNN Horror Stories
DNN-Connect 2019: DNN Horror StoriesWill Strohl
 
DNN-Connect 2019: Build a Module in Minutes
DNN-Connect 2019: Build a Module in MinutesDNN-Connect 2019: Build a Module in Minutes
DNN-Connect 2019: Build a Module in MinutesWill Strohl
 
DNN Awareness Meeting May 2019
DNN Awareness Meeting May 2019DNN Awareness Meeting May 2019
DNN Awareness Meeting May 2019Will Strohl
 
DNN Awareness Meeting April 2019
DNN Awareness Meeting April 2019DNN Awareness Meeting April 2019
DNN Awareness Meeting April 2019Will Strohl
 
DNN Awareness Meeting March 2019
DNN Awareness Meeting March 2019DNN Awareness Meeting March 2019
DNN Awareness Meeting March 2019Will Strohl
 
DNN Awareness Meeting February 2019
DNN Awareness Meeting February 2019DNN Awareness Meeting February 2019
DNN Awareness Meeting February 2019Will Strohl
 
DNN Awareness Meeting January 2019
DNN Awareness Meeting January 2019DNN Awareness Meeting January 2019
DNN Awareness Meeting January 2019Will Strohl
 
DNN Awareness Meeting December 2018
DNN Awareness Meeting December 2018DNN Awareness Meeting December 2018
DNN Awareness Meeting December 2018Will Strohl
 
DNN Upgrades Made Simple (DNN Summit 2019)
DNN Upgrades Made Simple (DNN Summit 2019)DNN Upgrades Made Simple (DNN Summit 2019)
DNN Upgrades Made Simple (DNN Summit 2019)Will Strohl
 
DNN Awareness EAG Meeting September 2018
DNN Awareness EAG Meeting September 2018DNN Awareness EAG Meeting September 2018
DNN Awareness EAG Meeting September 2018Will Strohl
 
DNN Awareness EAG Meeting August 2018
DNN Awareness EAG Meeting August 2018DNN Awareness EAG Meeting August 2018
DNN Awareness EAG Meeting August 2018Will Strohl
 
June 2018 DNN Awareness Group Meeting
June 2018 DNN Awareness Group MeetingJune 2018 DNN Awareness Group Meeting
June 2018 DNN Awareness Group MeetingWill Strohl
 
Evaluating Extensions: A Comprehensive Guide to Keeping Your Site Clean
Evaluating Extensions: A Comprehensive Guide to Keeping Your Site CleanEvaluating Extensions: A Comprehensive Guide to Keeping Your Site Clean
Evaluating Extensions: A Comprehensive Guide to Keeping Your Site CleanWill Strohl
 
Build a DNN Module in Minutes
Build a DNN Module in MinutesBuild a DNN Module in Minutes
Build a DNN Module in MinutesWill Strohl
 
DNN Database Tips & Tricks
DNN Database Tips & TricksDNN Database Tips & Tricks
DNN Database Tips & TricksWill Strohl
 

Más de Will Strohl (20)

DNN Community Newsletter: An In-Person Review of Recent Open-Source Activity
DNN Community Newsletter: An In-Person Review of Recent Open-Source ActivityDNN Community Newsletter: An In-Person Review of Recent Open-Source Activity
DNN Community Newsletter: An In-Person Review of Recent Open-Source Activity
 
Unveiling the Secrets of Software Company Transitions: Navigating the Path to...
Unveiling the Secrets of Software Company Transitions: Navigating the Path to...Unveiling the Secrets of Software Company Transitions: Navigating the Path to...
Unveiling the Secrets of Software Company Transitions: Navigating the Path to...
 
DNN CMS Awareness Group Meeting: December 2020
DNN CMS Awareness Group Meeting: December 2020DNN CMS Awareness Group Meeting: December 2020
DNN CMS Awareness Group Meeting: December 2020
 
Tips & Tricks: Working from Home and Staying Productive
Tips & Tricks: Working from Home and Staying ProductiveTips & Tricks: Working from Home and Staying Productive
Tips & Tricks: Working from Home and Staying Productive
 
DNN Awareness Meeting July 2019
DNN Awareness Meeting July 2019DNN Awareness Meeting July 2019
DNN Awareness Meeting July 2019
 
DNN-Connect 2019: DNN Horror Stories
DNN-Connect 2019: DNN Horror StoriesDNN-Connect 2019: DNN Horror Stories
DNN-Connect 2019: DNN Horror Stories
 
DNN-Connect 2019: Build a Module in Minutes
DNN-Connect 2019: Build a Module in MinutesDNN-Connect 2019: Build a Module in Minutes
DNN-Connect 2019: Build a Module in Minutes
 
DNN Awareness Meeting May 2019
DNN Awareness Meeting May 2019DNN Awareness Meeting May 2019
DNN Awareness Meeting May 2019
 
DNN Awareness Meeting April 2019
DNN Awareness Meeting April 2019DNN Awareness Meeting April 2019
DNN Awareness Meeting April 2019
 
DNN Awareness Meeting March 2019
DNN Awareness Meeting March 2019DNN Awareness Meeting March 2019
DNN Awareness Meeting March 2019
 
DNN Awareness Meeting February 2019
DNN Awareness Meeting February 2019DNN Awareness Meeting February 2019
DNN Awareness Meeting February 2019
 
DNN Awareness Meeting January 2019
DNN Awareness Meeting January 2019DNN Awareness Meeting January 2019
DNN Awareness Meeting January 2019
 
DNN Awareness Meeting December 2018
DNN Awareness Meeting December 2018DNN Awareness Meeting December 2018
DNN Awareness Meeting December 2018
 
DNN Upgrades Made Simple (DNN Summit 2019)
DNN Upgrades Made Simple (DNN Summit 2019)DNN Upgrades Made Simple (DNN Summit 2019)
DNN Upgrades Made Simple (DNN Summit 2019)
 
DNN Awareness EAG Meeting September 2018
DNN Awareness EAG Meeting September 2018DNN Awareness EAG Meeting September 2018
DNN Awareness EAG Meeting September 2018
 
DNN Awareness EAG Meeting August 2018
DNN Awareness EAG Meeting August 2018DNN Awareness EAG Meeting August 2018
DNN Awareness EAG Meeting August 2018
 
June 2018 DNN Awareness Group Meeting
June 2018 DNN Awareness Group MeetingJune 2018 DNN Awareness Group Meeting
June 2018 DNN Awareness Group Meeting
 
Evaluating Extensions: A Comprehensive Guide to Keeping Your Site Clean
Evaluating Extensions: A Comprehensive Guide to Keeping Your Site CleanEvaluating Extensions: A Comprehensive Guide to Keeping Your Site Clean
Evaluating Extensions: A Comprehensive Guide to Keeping Your Site Clean
 
Build a DNN Module in Minutes
Build a DNN Module in MinutesBuild a DNN Module in Minutes
Build a DNN Module in Minutes
 
DNN Database Tips & Tricks
DNN Database Tips & TricksDNN Database Tips & Tricks
DNN Database Tips & Tricks
 

Último

Call Us ➥9654467111▻Call Girls In Delhi NCR
Call Us ➥9654467111▻Call Girls In Delhi NCRCall Us ➥9654467111▻Call Girls In Delhi NCR
Call Us ➥9654467111▻Call Girls In Delhi NCRSapana Sha
 
Brand experience Dream Center Peoria Presentation.pdf
Brand experience Dream Center Peoria Presentation.pdfBrand experience Dream Center Peoria Presentation.pdf
Brand experience Dream Center Peoria Presentation.pdftbatkhuu1
 
Social Samosa Guidebook for SAMMIES 2024.pdf
Social Samosa Guidebook for SAMMIES 2024.pdfSocial Samosa Guidebook for SAMMIES 2024.pdf
Social Samosa Guidebook for SAMMIES 2024.pdfSocial Samosa
 
Five Essential Tools for International SEO - Natalia Witczyk - SearchNorwich 15
Five Essential Tools for International SEO - Natalia Witczyk - SearchNorwich 15Five Essential Tools for International SEO - Natalia Witczyk - SearchNorwich 15
Five Essential Tools for International SEO - Natalia Witczyk - SearchNorwich 15SearchNorwich
 
Google 3rd-Party Cookie Deprecation [Update] + 5 Best Strategies
Google 3rd-Party Cookie Deprecation [Update] + 5 Best StrategiesGoogle 3rd-Party Cookie Deprecation [Update] + 5 Best Strategies
Google 3rd-Party Cookie Deprecation [Update] + 5 Best StrategiesSearch Engine Journal
 
Situation Analysis | Management Company.
Situation Analysis | Management Company.Situation Analysis | Management Company.
Situation Analysis | Management Company.DanielaQuiroz63
 
Defining Marketing for the 21st Century,kotler
Defining Marketing for the 21st Century,kotlerDefining Marketing for the 21st Century,kotler
Defining Marketing for the 21st Century,kotlerAmirNasiruog
 
Branding strategies of new company .pptx
Branding strategies of new company .pptxBranding strategies of new company .pptx
Branding strategies of new company .pptxVikasTiwari846641
 
Kraft Mac and Cheese campaign presentation
Kraft Mac and Cheese campaign presentationKraft Mac and Cheese campaign presentation
Kraft Mac and Cheese campaign presentationtbatkhuu1
 
Local SEO Domination: Put your business at the forefront of local searches!
Local SEO Domination:  Put your business at the forefront of local searches!Local SEO Domination:  Put your business at the forefront of local searches!
Local SEO Domination: Put your business at the forefront of local searches!dstvtechnician
 
The Skin Games 2024 25 - Sponsorship Deck
The Skin Games 2024 25 - Sponsorship DeckThe Skin Games 2024 25 - Sponsorship Deck
The Skin Games 2024 25 - Sponsorship DeckToluwanimi Balogun
 
Cost-effective tactics for navigating CPC surges
Cost-effective tactics for navigating CPC surgesCost-effective tactics for navigating CPC surges
Cost-effective tactics for navigating CPC surgesPushON Ltd
 
Brand experience Peoria City Soccer Presentation.pdf
Brand experience Peoria City Soccer Presentation.pdfBrand experience Peoria City Soccer Presentation.pdf
Brand experience Peoria City Soccer Presentation.pdftbatkhuu1
 
Factors-Influencing-Branding-Strategies.pptx
Factors-Influencing-Branding-Strategies.pptxFactors-Influencing-Branding-Strategies.pptx
Factors-Influencing-Branding-Strategies.pptxVikasTiwari846641
 
April 2024 - VBOUT Partners Meeting Group
April 2024 - VBOUT Partners Meeting GroupApril 2024 - VBOUT Partners Meeting Group
April 2024 - VBOUT Partners Meeting GroupVbout.com
 

Último (20)

Call Us ➥9654467111▻Call Girls In Delhi NCR
Call Us ➥9654467111▻Call Girls In Delhi NCRCall Us ➥9654467111▻Call Girls In Delhi NCR
Call Us ➥9654467111▻Call Girls In Delhi NCR
 
Brand experience Dream Center Peoria Presentation.pdf
Brand experience Dream Center Peoria Presentation.pdfBrand experience Dream Center Peoria Presentation.pdf
Brand experience Dream Center Peoria Presentation.pdf
 
Social Samosa Guidebook for SAMMIES 2024.pdf
Social Samosa Guidebook for SAMMIES 2024.pdfSocial Samosa Guidebook for SAMMIES 2024.pdf
Social Samosa Guidebook for SAMMIES 2024.pdf
 
Five Essential Tools for International SEO - Natalia Witczyk - SearchNorwich 15
Five Essential Tools for International SEO - Natalia Witczyk - SearchNorwich 15Five Essential Tools for International SEO - Natalia Witczyk - SearchNorwich 15
Five Essential Tools for International SEO - Natalia Witczyk - SearchNorwich 15
 
Google 3rd-Party Cookie Deprecation [Update] + 5 Best Strategies
Google 3rd-Party Cookie Deprecation [Update] + 5 Best StrategiesGoogle 3rd-Party Cookie Deprecation [Update] + 5 Best Strategies
Google 3rd-Party Cookie Deprecation [Update] + 5 Best Strategies
 
Situation Analysis | Management Company.
Situation Analysis | Management Company.Situation Analysis | Management Company.
Situation Analysis | Management Company.
 
Creator Influencer Strategy Master Class - Corinne Rose Guirgis
Creator Influencer Strategy Master Class - Corinne Rose GuirgisCreator Influencer Strategy Master Class - Corinne Rose Guirgis
Creator Influencer Strategy Master Class - Corinne Rose Guirgis
 
Generative AI Master Class - Generative AI, Unleash Creative Opportunity - Pe...
Generative AI Master Class - Generative AI, Unleash Creative Opportunity - Pe...Generative AI Master Class - Generative AI, Unleash Creative Opportunity - Pe...
Generative AI Master Class - Generative AI, Unleash Creative Opportunity - Pe...
 
Defining Marketing for the 21st Century,kotler
Defining Marketing for the 21st Century,kotlerDefining Marketing for the 21st Century,kotler
Defining Marketing for the 21st Century,kotler
 
Top 5 Breakthrough AI Innovations Elevating Content Creation and Personalizat...
Top 5 Breakthrough AI Innovations Elevating Content Creation and Personalizat...Top 5 Breakthrough AI Innovations Elevating Content Creation and Personalizat...
Top 5 Breakthrough AI Innovations Elevating Content Creation and Personalizat...
 
Branding strategies of new company .pptx
Branding strategies of new company .pptxBranding strategies of new company .pptx
Branding strategies of new company .pptx
 
Kraft Mac and Cheese campaign presentation
Kraft Mac and Cheese campaign presentationKraft Mac and Cheese campaign presentation
Kraft Mac and Cheese campaign presentation
 
Local SEO Domination: Put your business at the forefront of local searches!
Local SEO Domination:  Put your business at the forefront of local searches!Local SEO Domination:  Put your business at the forefront of local searches!
Local SEO Domination: Put your business at the forefront of local searches!
 
Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel LeminTurn Digital Reputation Threats into Offense Tactics - Daniel Lemin
Turn Digital Reputation Threats into Offense Tactics - Daniel Lemin
 
The Skin Games 2024 25 - Sponsorship Deck
The Skin Games 2024 25 - Sponsorship DeckThe Skin Games 2024 25 - Sponsorship Deck
The Skin Games 2024 25 - Sponsorship Deck
 
Cost-effective tactics for navigating CPC surges
Cost-effective tactics for navigating CPC surgesCost-effective tactics for navigating CPC surges
Cost-effective tactics for navigating CPC surges
 
Brand experience Peoria City Soccer Presentation.pdf
Brand experience Peoria City Soccer Presentation.pdfBrand experience Peoria City Soccer Presentation.pdf
Brand experience Peoria City Soccer Presentation.pdf
 
Factors-Influencing-Branding-Strategies.pptx
Factors-Influencing-Branding-Strategies.pptxFactors-Influencing-Branding-Strategies.pptx
Factors-Influencing-Branding-Strategies.pptx
 
Brand Strategy Master Class - Juntae DeLane
Brand Strategy Master Class - Juntae DeLaneBrand Strategy Master Class - Juntae DeLane
Brand Strategy Master Class - Juntae DeLane
 
April 2024 - VBOUT Partners Meeting Group
April 2024 - VBOUT Partners Meeting GroupApril 2024 - VBOUT Partners Meeting Group
April 2024 - VBOUT Partners Meeting Group
 

DNN Summit: Robots.txt & Multi-Site DNN Instances

  • 1. Robots.txt & Multi-Site DNN Instances A Primer on SEO & Robots.txt in DNN 9.8+
  • 2. Thank You to Our Sponsors!
  • 3. Will Strohl • Using DNN since 2003 • CEO & Founder, Upendo Ventures • Former DNN Corp Employee • Former CEO, Hotcakes Commerce (2013-2017) • DNN MVP Since 2013 • Leader of the Awareness Group • DNN Author & Technical Editor • Founder of Day of DNN & DNNCon • Maintainer of 40+ Open-Source Projects on GitHub @WillStrohl WillStrohl.com
  • 4. Upendo Ventures • DNN Platform & Evoq • E-Commerce • Implementations • Integrations • Custom Modules • Upgrades & Edition Migrations • Custom Themes • Training • Fixing Things Other People Did ;) @UpendoVentures UpendoVentures.com
  • 5. Robots.txt & Multi-Site DNN Instances A Primer on SEO & Robots.txt in DNN 9.8+
  • 6. Objectives I Hope You Learn… • The Basics of SEO • How to Manage Robots.txt • The Do’s & Don’ts of Robots.txt • How to Have More than One Robots.txt in Multi-Portal Environments Prerequisites • You Can Spell SEO ;) • General Knowledge of How SEO Works • General Understanding of Your Website Implementation • General Understanding of How to Administrate DNN
  • 7. What is SEO? Search Engine Optimization (“SEO”) The process of maximizing the number of visitors to a particular website by ensuring that the site appears high on the list of results returned by a search engine.
  • 8. Common SEO Techniques • Digital Marketing Strategy • Content Creation • Regular Content Updates • Visitor Engagement • Offsite Content & Links • Multiple Mediums • Multiple Channels • Use HTTPS for All Requests • Generate a Video Playlist • Reduce 404 Errors • Decrease Page Load Times • Keyword Research • Keyword Acquisition • Refine Targeted Content • Use Schema Markup • Properly Implement Semantic Markup • Optimize for Voice Search
  • 9. Common SEO Techniques Keep the Search Engines & Crawlers Informed
  • 10. Page Meta Data <meta name="robots" content="noindex" /> <meta name="robots" content="nofollow" /> <meta name="robots" content="noindex,nofollow" /> Default: Index, Follow
  • 11. Page Meta Data <meta id="MetaRobots" name="robots" content="INDEX, FOLLOW"> <meta id="MetaRobots" name="robots" content="NOINDEX, NOFOLLOW">
  • 12. What is Robots.txt? The robots exclusion standard, also known as the robots exclusion protocol or simply robots.txt, is a standard used by websites to communicate with web crawlers and other web robots. The standard specifies how to inform the web robot about which areas of the website should not be processed or scanned.
  • 13. User-agent User-agent: [bot identifier/crawler name] Specifies a bot or crawler to instruct. User-agent: msnbot User-agent: Googlebot User-agent: Slurp User-agent: *
  • 14. Allow/Disallow Allow: [path to crawl] Disallow: [path to ignore] Specifies a path where crawlers are or are not allowed to go. Allow: / Disallow: /activity-feed/
  • 15. Crawl-delay Crawl-delay: [milliseconds] Indicates a period of time to wait between page requests. Crawl-delay: 120
  • 16. Sitemap Sitemap: [path to sitemap] Allows you to specify the location of one or more sitemaps for the search engine to follow. Sitemap: https://dnncommunity.org/sitemap.xml
  • 18. Pattern Matching * is a wildcard pattern, matching any character $ is the end of the URL Allow: *.html Disallow: *.php Allow: *.aspx$
  • 20. Robots.txt in DNN • Included since DNN 7.x • Update for Every Instance • Difficult to Support Multi-Site • Used to Require Direct File System Access • DnnSoftware.com has Good Examples
  • 21. Multi-Site Robots.txt in DNN • Requires Rewrite Rules Module • https://www.iis.net/downloads/microsoft/url-rewrite • Create a Robots.txt for Each Site • example.com.robots.txt • Add a Rewrite Rule for Each Site • example.com.robots.txt becomes robots.txt
  • 23. Demo Let’s See This Stuff in Action!
  • 24. Thank You to Our Sponsors!
  • 25. Upendo Ventures Summary • SEO important on all public sites • There are many ways to improve SEO (all are important) • DNN makes all of these easy • Robots.txt is fully supported • Robots.txt files can be edited directly in the Config Manager • Be sure to throttle bots using Crawl- delay • Multi-site instances require some more configuration • Review robots.txt after any major roll-out We use technology to help your business change people’s lives UpendoVentures.com solutions@upendoventures.com (650) 381-9160 548 Market St. #65401 San Francisco, CA 94104

Notas del editor

  1. https://en.wikipedia.org/wiki/Robots_exclusion_standard https://developers.google.com/search/docs/advanced/robots/intro
  2. Generally, it’s considered a best practice to have this at the end of the robots.txt file.
  3. Always been able to add a robots.txt to the root of the website. Now able to edit the robots.txt directly in the DNN Config Manager as of version 9.8.
  4. Where do I edit the meta tags for a specific page? Where is the Config Manager and how do I use it?