protecting web servers from robots

[chris at sparkie.osl.state.or.us]

Can anyone provide information about how to protect web servers from the
robots that roam the internet, such as Altavista, Lycos, etc. I have several
Unix servers and  a Mac server. I know that I have heard of a way to do
this, but can't remember what it was.

Christopher Adams
Oregon State Library
chris at sparkie.osl.state.or.us