more on robot exclusion

[george at library.caltech.edu]

Jakob Nielsen touches on the robot exclusion protocol, including the correct
META tag implementation, in his current Alertbox column, Deep linking is
good linking <http://www.useit.com/alertbox/20020303.html>.  Included is
some guidance on what kinds of pages should (and should not) use robot
exclusion.

On the lighter side of the robot protocol, enjoy
<http://ftrain.com/robot_exclusion_protocol.html>.

George S. Porter
Sherman Fairchild Library of Engineering & Applied Science
Caltech, 1-43
Pasadena, CA  91125-4300
Telephone (626) 395-3409 Fax (626) 431-2681