Google Prioritizes Links in Partial Sitemaps for Crawling

With so many sites autogenerating sitemaps, we don’t hear a lot about custom sitemaps or sitemaps where only a part of a site is on it.  In yesterday’s keynote with Stonetemple, Gary Illyes from Google brought up an interesting aspect with sitemaps and how Google prioritizes crawling.

If you use a sitemap that is not autogenerated to automatically include all the URLs on the site, then Google will consider the pages that are in the sitemap as more important and Google will crawl it sooner.

Here is what he said:

If this URL is in a sitemap, we will probably want to crawl it sooner or more often because you deemed that page more important by putting it into a sitemap.  We can also learn that this might not be true when sitemaps are automatically generated, like for every single URL entry in the sitemap.

There are also sites where they have all their URLs divided between multiple sitemaps, as a strategy for debugging indexing issues.  It can be easier to track down pages or page types that aren’t being indexed as expected when you have your sitemaps divided by page type or site section or category.  But in this case because all pages are still indexed, Google likely wouldn’t be able to give higher preference to any, since all the URLs are still contained in the sitemaps.

That said, unless your server is severely impacted by Googlebot crawling – when Googlebot will adjust the crawl rate on the site anyway – it might not make that much of a difference.  But if you are having problems getting your most important pages crawled as regularly as you’d like, it could be a strategy to reduce the sitemap to the site’s most important pages instead of automatically including them all.

The post Google Prioritizes Links in Partial Sitemaps for Crawling appeared first on The SEM Post.

from The SEM Post http://ift.tt/1WNsxXo
http://ift.tt/1WNszOJ via IFTTT