[WEB4LIB] Re: Z39.50 Discussion on Web4Lib

[Sebastian Hammer]

At 23:45 11-02-01 +0000, Matthew Dovey wrote:

>There is another scalability issue - if you are searching 500 targets you
>are like to get 500 times as many hits, so the user needs to put in more
>specific queries. Also the necessity for dedup begins to surface.

The first issue is really a general union catalogue issue, not a parallel 
searching issue as such. Whenever you build composite catalogs, you're 
potentially increasing the level of "noise". The second issue is more 
serious. From a performance standpoint, it's hard to imagine dedup without 
predictable sorting on the server side, to avoid having to download 
potentially huge result sets. But rules for sorting has not yet made it 
into many of the national or international library profiles for Z39.50. 
Sorting, obviously -- as well as dedup in general -- is complicated by 
issues like character sets and name or title normalisation schemes. Anyone 
who has worked with union cat systems know that deduplication is a 
non-trivial process... in the Z39.50 model, it essentially has to happen on 
the fly - preferably without sacrificing response speed. It certainly keeps 
things interesting.

There is an interesting twist to this. Suppose you use a centralised union 
cat to do your general searching, and find the exact title you want. Then 
you use Z39.50 only to find libraries which might potentially be able to 
supply the thing. You can then launch a massively parallel search which is 
very exact - say, a title/author or ISBN search, which might only return 
one or two hits at the most. There is no dedup issue, and the load on the 
target systems is much reduced. On the other hand, the user gets the 
responsiveness of a physical union catalgue, combined with up-to-the second 
holding (and perhaps even circulation) information from a large number of 
libraries.

To answer the original question - I'd think Z39.50 is far from dead.. but 
it is likely that we have not yet figured out exactly how to use it to best 
advantage.

>However, the essential point is that these are no Z39.50 issues, but general
>issues which infortunately this use of Z39.50 tends to expose...

As would any protocol which attempted to do the same thing. It seems to me 
the only real alternative would be something on the line of OAI idea, which 
could simplify the maintenance of *physical* union catalogues. But that 
obviously comes with its own set of challenges.

--Sebastian
--
Sebastian Hammer        <quinn at indexdata.dk>            Index Data ApS
Ph.: +45 3341 0100    <http://www.indexdata.dk>    Fax: +45 3341 0101