inconsistencies web search performance

[Booher, Craig]

	On November 7, Ernest Perez wrote:

	<snip>
	  ** We can quibble about Search Engine X getting 22,000 hits
vs. Search
	Engine Y getting 14,000 hits. So what, do you plan on examining
all of
	them? These systems operate on presentation ranking algorithms,
and they
	attempt to present relevant items first.  _I'm_ certainly not
going to
	look at item #4,763, and I presume you aren't going to either.
	<snip>
	 
	Now we have reached the second conundrum faced by users of
Internet search engines - relevancy ranking.  While our knowledge of the
search engine details may be unacceptable, we know even less about their
relevancy ranking algorithms.  With what confidence can we state that
(using the example provided) item #4,763 is less relevant than
previously presented items?  If the search engine is not conducting the
search using our syntax (based on our contextual understanding of our
information needs) how can we rely on an unknown relevancy ranking
algorithm to present the information most relevant to our needs in
priority order?  As we call for more complete disclosure of search
engine operations, let's also include a request for more extensive
descriptions of these relevancy ranking algorithms.