protecting web servers from robots

[Jian Liu]

Check the following:

http://info.webcrawler.com/mak/projects/robots/norobots.html

Jian
Indiana University Libraries

> 
> Can anyone provide information about how to protect web servers from the
> robots that roam the internet, such as Altavista, Lycos, etc. I have several
> Unix servers and  a Mac server. I know that I have heard of a way to do
> this, but can't remember what it was.
> 
> Christopher Adams
> Oregon State Library
> chris at sparkie.osl.state.or.us
> 
>