Maneuver crawler to your will

control

I was reviewing one of our client websites and found few issues which are considered to be SEO pitfalls.  Would like to share this, I am assuming that most of us already know this, but, in this myriad of things we need to keep a tap on, it is almost necessary to remind ourselves just how important SEO aspect for the website you will build actually is.   Sitemap is almost an inherent choice and much needed one to submit to the search engine giants in some way or choice.

But, it is essential to remember what not to push to sitemap as crawler is busy doing it’s thing and indexing every bit of your fresh content as much as it can.   Few things to always remember –

  1. Ensure to not add any urls that either do not have any content or could lead to 404.  This is very essential to make sure we are not asking crawler to even for a milli-second think about pages that are not important and this will help ensure you don’t set up your own pitfall towards having a red mark on Google for instance.  You can do this by providing a way to either exclude content or templates when sitemap is being generated, easy enough.  But, is often missed. 🙂
  2. Now, if you are using any folders or intermediate content for just solely organizational purpose, ensure you have proper redirects in play to make sure if some one is either being smart or received an improper url, your set up does a terrific job of placing the end user where you wish him/her to be looking at when they pull up your site.  Magic, yeah. lol.  For the other side of users which is your content authors, ensure you have proper insert options and templates filled in with beautiful presentation that would then take care of all in the background keeping content authoring simple and easy peasy.

You do not have such fancy sitemap generation on your sitecore instance? Look the following options which are my favorite.  Setting these up should be real simple, but, as always tricky part is maintaining the solution and training some one who uses it to know these things that they can do to keep crawlers in check.

References / Suggestions 

https://marketplace.sitecore.net/Modules/S/SitemapXml.aspx

https://marketplace.sitecore.net/en/Modules/XML_Sitemap_Generator.aspx

https://github.com/JimmieOverby/SitecoreSitemapXML    — My Personal Favorite with more customization

Tons of other modules on market place, explore more:

https://marketplace.sitecore.net/SearchResults#query=sitemap