URL checker question

Charles Blair chas at nirvana.lib.uchicago.edu
Mon Jan 22 16:05:26 EST 1996


After some looking around, we wrote our own URL checker, or program to
detect stale URLs in Web pages and compose and mail reports to web
page maintainers detailing the kind of error and recommended
action. For a web our size, we had to deal with some scalability
issues. For example, if you're trying to check over a 1000 URLs in a
reasonable timeframe, you have to manage the business intelligently,
so that several timeouts don't protract the job unnecessarily (which
would be the case, for example, if one assembled then checked all the
URLs sequentially).

Anyway, I hear now that OCLC has written such a program. I don't find
mention of it in my archives of this group's discussions, nor can I
find reference to it on OCLC's own site.

Has anyone else heard of this program? If so, could someone point me
to more information about it so I can compare its functionality to
what we wrote in-house?  E-mail responses preferred.

Thanks.



More information about the Web4lib mailing list