[WEB4LIB] Re: Forcing a click-thru legal agreement (no robots,

[Ryan Eby]

I agree that I don't think it requires a click-thru. I don't think
many users read agreements and it has the potential of frustrating
people that just want to use the resource in a legitimate way. I can
see the point if you need a legal document or something so you have a
method of recourse. For normal users I think a small box near the top
or the search box (or on the download page, etc.) that says "By using
this resource you agree not to....). By being front and center it
should raise awareness while not becoming a large barrier for people
using it in a normal matter.

You may also look as making a robots.txt file on your web server (if
it is hosted on it) that will help prevent legitimate robots from
crawling the database. For robots that don't follow the rules you may
need to ban IPs, etc. if they become a large problem (hammering the
server, etc.). I don't really know how bad your problem is so I can't
really say more than that. Here's a tutorial on writing a robots.txt
file:

http://www.searchengineworld.com/robots/robots_tutorial.htm

Ryan Eby

On Tue, 30 Nov 2004 15:13:41 -0800 (PST), Genny Engel
<gengel at sonoma.lib.ca.us> wrote:
> Actually, I have spent my time since starting here attempting to REDUCE
> the number of clicks required to get from our home page to actual
> information.  Luckily none of our database vendors has ever brought up
> this issue and I hope they never do.  Since all our vendor databases
> require a library card number to be entered into a form, most robots
> wouldn't get into the databases, unless they were specifically crafted
> to use a stolen library card number.
> 
> Are you looking into this because you have actual evidence of robots
> harvesting the databases?  Or just as a CYA?  I would have thought if
> the vendors were actually getting twitchy about this issue, they would
> have started putting up these click-through pages on the databases
> themselves.
> 
> Genny Engel
> Internet Librarian
> Sonoma County Library
> gengel at sonoma.lib.ca.us
> 707 545-0831 x581
>