Know any good spiders?

[Michael Alan Dorman]

On Thu, 14 Dec 1995, Frank Cervone wrote:
> Does anyone out there have experience with spider (or spider-like)
> programs that validate the URLs on your site's pages? Any info will be
> appreciated. Even if you are using a manual system, I'd like to know the
> implementation details. Please respond to me directly, and I'll summarize
> the results for the list.

Well, it's not what I think of as a spider---I think of a spider as
something that runs an indexer, though that may just be small-mindedness
on my part---but you can check out: 

http://lot49.med.miami.edu/~mdorman/checklinks.html

For a look at the checker I have created.  It's in perl, and frankly, I
don't know if it'll run on a non-Unix system, so it might not work for
you, since you don't mention what platform your server is on.

It's designed to check integrity on a local scale (within documents ona
single server), though it will even check links that travel off-site if 
you wish. 

It's free software.  Pick up a copy if you wish. 

Mike.
--
Michael Alan Dorman                                   Head of Systems
mdorman at caldmed.med.miami.edu           Louis Calder Memorial Library
(305) 243-5530                     University of Miami Medical School