*implementing* search engines for a campus

[Prentiss Riddle]

John Ockerbloom, who reads WEB4LIB via a locally gated newsgroup
and therefore can't post to it, asked me to forward the following
message for him, along with my reply.

-- Prentiss Riddle ("aprendiz de todo, maestro de nada") riddle at rice.edu
-- RiceInfo Administrator, Rice University / http://is.rice.edu/~riddle
-- Home office: 2002-A Guadalupe St. #285, Austin, TX 78705 / 512-323-0708

--------------------------------------------------------------------------

From: spok at GS1.SP.CS.CMU.EDU
Subject: Re: *implementing* search engines for a campus
Organization: School of Computer Science, Carnegie Mellon
Date: Tue, 9 Jan 96 14:29:19 EST

Prentiss Riddle wrote:
>A bigger problem for us is access control and the prevention of
>"leakage".....  Through what
>seems to me to be poor foresight on the part of protocol designers,
>protocols like HTTP, gopher, FTP, WAIS, or you-name-it have no
>mechanism for a search robot or a proxy server or other gateway to say,
>"treat me like an unprivileged user at some off-campus site".

We encountered that problem too.  Our approach was to extend the
HTTP protocol (in a way compatible with HTTP/1.0) to let the
proxy server say "I'm doing this for third party X", or "I will
give this information to anyone (or anyone in a certain domain)."

Then we just have the HTTP server check access both for the proxy
machine and for whoever the proxy machine might be working for.

I believe something like this eventually needs to be part of the
standard HTTP protocol.  We plan to announce our standard more widely
once the term starts and I get more feedback on it.  But I'd be happy
to send you details about the protocol we use.

John Mark Ockerbloom

--------------------------------------------------------------------------

From: riddle at rice.edu
Subject: Re: *implementing* search engines for a campus
To: spok at GS1.SP.CS.CMU.EDU
Date: Tue, 9 Jan 1996 15:50:09 -0600 (CST)

I applaud your efforts.  However, I don't see us making use of your
software while it's still in the experimental stage.  For one thing, it
wouldn't solve my immediate problem until and unless it were pretty
nearly universal -- *every* HTTP server on campus would have to have a
shot at running your extended protocol in order for me to build a
reasonably leak-proof web index from within the campus.

That said, I hope your idea is quickly accepted by the web community.
Thanks for doing something about the problem instead of just moaning
about it like me. :-)

-- Prentiss Riddle ("aprendiz de todo, maestro de nada") riddle at rice.edu
-- RiceInfo Administrator, Rice University / http://is.rice.edu/~riddle
-- Home office: 2002-A Guadalupe St. #285, Austin, TX 78705 / 512-323-0708