Agents (Was Re: Students use of search engines.)

[Nick Arnett]

>The Holy Grail of electronic information retrieval is precision natural
>language searching.

Certainly many people think so, but at least some of who build search
products see it as much more complex.  Natural language is just one issue.
Our product development includes other things such as clustering, smarter
query expansion and much more.  Ultimately, the search engine's results are
simply the software's opinion of which documents matched the words or
phrases that you supplied to it.  When you realize that it's that
subjective (which must include an appreciation for our lack of
understanding the subtleties and complexities of languages) you realize
that for a search engine to do what some imagine is around the corner,
they'll have to be much smarter than people.  No amount of brute force
computing power gets you there.

And even when you have good search algorithms, you have to be able to scale
from the desktop to the server, gateway to many document storage systems
(files, Web, database, etc.), filter the words out of many document
formats, and give people access to search and agents from the interfaces
they normally use (the Web, groupware, e-mail, etc.).

>If you want to see the future, check out an Article entitled 'Agents of
>change' in the May 96 issue of Internet World. I think that eventually no
>one will ever touch a search engine. All searching will be done on your
>behalf by agents.

This begs the definition of agent, doesn't it?  Right now, it generally
refers to persistent searches.  In other words, you've defined something
that you're interested in and you tell the agent software to keep an eye
out for it.  The big problem there is it can't find anything that you
didn't tell it to watch for, so you don't see  new things... unless you
make the leap to searching and watching meta-information, such as the sort
of thing being done by Firefly and others.

Nick