Organizing Web Information (was: Something Missing)

[Edward Wigg]

Martin Cohen wrote in response to Temple Hoff:
>I'm sure that the beliefs / desires expressed are sincerely felt by many. 
>That is, "no one wants an index of sources, they just want the answer...".
>And that soon it will be possible to provide the answer by some technical 
>means which does not involve indexing, cataloging, classifying, 
>abstracting, or other intellectual skills. It is a chimera.

Well it's all a chimera; all the methods are bound to fail! 

Librarians and others will toil valiantly with MARC, Z39.50, and whatever
else already exists and time brings, to impose structure and order and make
finding the useful from among the questionable easier. It will be a valiant
effort, with many good things coming from it, but it will never "succeed."
Even if it could in a stable environment, time and technology will move the
problem out from under the effort. It has happened before: the
encyclopedists of the 18th century wanted to collect all human knowledge in
a multi-volume set of books. They were doing it (not coincidentally) at the
same time that the volume what was known was growing exponentially, making
their original intention futile. But that doesn't mean that encyclopedias
are not some of the most useful reference books, routinely consulted first
even now that the sum of human knowledge won't even fit into a single library.

Technology will improve wonderfully, beyond all expectations in some areas,
but many things will remain unreachable dreams. Frankly I don't expect
computers to be able to do anything particularly useful when you "type in
(or speak) whole paragraphs or dialogs of explanation about what you want,"
any time real soon, probably not within my lifetime (say the next 40-50
years if I'm statistically average). In part this is because people often
can't express what they want in the first place, so to do the job there will
either have to be a long session of questions refining what is actually
wanted (possibly useful, but most users won't sit still for it), or the
computer will have to pass the Turing test in a nonrestricted domain (or it
could become clairvoyant :-). Expert systems have shown us that wonderful
things can be done within very limited domains, but we are talking
potentially about the whole of human knowledge -- a whole new ball game.

The commercial end of the business will give us instant access to what the
providers think we want, or more accurately what their advertisers think
will keep us looking at their ads. Whether this is on the Web or coming from
our set-top boxes, it will give us more than we dreamed possible, but much
of it will be irrelevant or misleading, unless there is more money to be
made by being complete and accurate (it's bound to be the case sometimes,
but don't expect it to be the general rule). It is generally agreed that
thirty channels of television often give you no more to watch than when
there were just three or four channels; don't think that 500 will be any
different. Video on demand will be great, but what will be available is what
will generate revenue: last year's blockbuster will be there but don't
expect the entire MGM film library real soon, much less every newsreel ever
shot. Searching for information will suffer from the same problems. The
interface will tend to favor the lowest common denominator -- maybe you can
do in depth research only using four buttons on your remote control, but it
will require a _lot_ of clicking and scrolling. A virtual reality interface
might be possible, but expect it to be used more for pornography than
searching distant libraries or knowledge bases. The results of commercial
development will be fun and rewarding, but anything but complete or
authoritative. Still we do all want entertainment of one sort or another, so
it will be enjoyable and often informative, even if the signal to noise
ratio is not too good.

This may all sound pessimistic, but really I don't think that it is. We
should all do what we're good at, striving to do it as well as possible; the
results will not be quite what we expect, and doubtless they will fall short
of our hopes in some respects, but the combination of what we all have done
will far exceed our hopes and dreams in unexpected and wonderful ways. We
should just not be too hung up on finding the right, or even worse, the one
way of doing things. Standards are useful, but a fully defined, unchanging
standard probably describes a moribund or obsolete technology -- if people
can't think of new ways of breaking the rules to improve things, they
probably aren't thinking very creatively.

In practical terms this means that librarians and others should not
surrender to those who believe that all things can be provided as long as
there is a small commercial message at the top; they should develop indexes
and other resources their way, following their own best practices and their
own best interest. But they should not be surprised or offended that there
are commercial sources out there getting most of the interest of the media,
after all the media is a commercial venture too! And yes people will often
not know/recognize that the pretty picture at the top is a commercial, or
that the ratings can be bought. There are probably people out there who
don't realize that the golden arches that loom in the distance as you drive
along the interstate are not put there as an exercise in altruism for the
hungry traveler, but is it important? Instead you can use a guide book to
find stoping places, but that is probably not an altruistic venture either.
For that matter public libraries (where you might stop and read the guide
book for free) are not really a form of altruism, but rather a public good
that it benefits us all to pay for collectively.

Just my random thoughts,

Edward
--------------------------------------------------------------
Edward Wigg                      "Just another guy, you know?"
Evanston Public Library             e-wigg at evanston.lib.il.us
Evanston, Illinois