protecting web servers from robots

[Mary-Ellen Mort]

I just finished reading Dr. Website's column in the June 3 edition of Web 
Week. He says you can create a file called robots.txt and place it at the 
root level of your server. He gives an example of how you can limit 
specific robots, or all robots, from certain sections of your server.

Unfortunately I don't see this particular question in Dr. Website's 
column ont he Web Week site:
http://www.webweek.com/drwebsite/index.html

Dr. Website recommends WWW Robots FAQ at:
http://info.webcrawler.com/mak/projects/robots/faq.html

HTH, 
Mary-Ellen Mort, JobSmart Project Director
http://jobsmart.org