[Web4lib] Federated Search versus Crawler or Spider

[Robert L. Balliot]

Greetings

It seems to me that the intent of a Federated
search is to provide a user interface that
brings together all disparate database results. If
it does not do that completely, then the results
are false.  Other information may exist and
the process must undergo re-search. 

You may create links on a website that describe
how to search each database independently based
on search criteria.  This becomes a Federated
search with the final results dependent on the search 
skills of the human middleperson.  Of course, other
than searcher skills and time, there is no additional
cost.

I think that the problem of federated searching is
being addressed backwards. The data must first fit
into standards which removes the ambiguity of the
search mechanism, rather than trying to create a
search mechanism that can account for all abiguities.

I think that Federated searching is worthwhile as
an academic study to try to achieve great results.
However, it is unclear to me that this cost should
be on the shoulders of the end user or the developers.

>From a broader, organizational approach that can
be undertaken by libraries, we need to push and
agree on standards that will allow for that development.


*************************************************
Robert L. Balliot
1-401-421-5763
Skype: RBalliot
Bristol, Rhode Island
http://oceanstatelibrarian.com/contact.htm
*************************************************
-----Original Message-----
From: web4lib-bounces at webjunction.org
[mailto:web4lib-bounces at webjunction.org] On Behalf Of Andrew Ashton
Sent: Tuesday, July 03, 2007 8:59 AM
To: Ross Singer; McIntyre, Ruth
Cc: web4lib at webjunction.org
Subject: RE: [Web4lib] Federated Search versus Crawler or Spider

 Ross Singer wrote:
"2) the onus is on the content providers to provide a standardized
search interface - you lose all control about what is indexed/how it's
indexed and how search results are presented"

This seems to be the kiss of death for any really useful Federated
search projects.  Remember the 
OCLC SiteSearch project?  We did a proof-of-concept test here at
Skidmore College and discovered quickly that the lack of vendor
standardization made it impractical.  Librarians are nothing if not
completists, and offering a Federated Search service that only covered
2/3 of our resources wasn't going to cut it.  Sure, you could buy a
Federated Search product and let the vendor worry about maintaining
access to the non-standardized targets, but any technology that
precludes community development ought to be considered dead in the
water.

--
Andrew Ashton
Systems Librarian
Scribner Library, Skidmore College
(518)580-5505