Controlling Bloglines crawler?
    Thomas Dowling 
    tdowling at ohiolink.edu
       
    Tue Nov  9 16:28:49 EST 2004
    
    
  
Web4Libbers--
Has anyone encountered a way to throttle (in either sense of the word!) 
the Bloglines RSS crawler?  We have a server with a large number of 
feeds.  While Bloglines doesn't fetch any individual feed more than once 
an hour, they tend to fire off requests for all of our feeds in rapid 
succession, creating bursts of heavy traffic that can do bad things if 
they come in during an already high load on the server.  Any chance 
they'd understand a 304 Not Modified HTTP status?
-- 
Thomas Dowling
tdowling at ohiolink.edu
    
    
More information about the Web4lib
mailing list