[Web4lib] Sitemap.xml

[Thomas Dowling]

On 3/30/2007 12:38 PM, VanderHart, Robert wrote:

> A speaker on SEO at the IA Summit earlier this week stated that it's
> very important to have a sitemap.xml file for your website to indicate
> to spiders how often to visit your site.  I know from reviewing our
> server access logs that spiders should request a robots.txt file before
> indexing a site, and when I grep the logs I see plenty of requests for
> that file.  But when I grep "sitemap.xml", I don't see a single request.
> 
> 
> So the question is, if a sitemap.xml file is so important, why aren't
> any spiders looking for the file?  I didn't raise the question to the
> speaker because I couldn't view our log files while I was at the Summit,
> so I wasn't certain whether we were getting any requests for sitemap.xml
> or not.


Unlike robots.txt, you have to explicitly tell the search engines about
your sitemap.xml files.


https://www.google.com/webmasters/tools/docs/en/sitemap-generator.html#submitting

https://siteexplorer.search.yahoo.com/submit


-- 
Thomas Dowling