[Web4lib] OPAC task success rates

[Emily Lynema]

What about considering benchmarks used by the search industry in general 
(not just library OPACs)? Or using the amount of time it would take the 
accomplish the same task in a well-known search tool like Amazon, Barnes 
& Noble, Wal-Mart, etc? I went to amazon.com and typed 'the red pony' 
(no quotation marks) in their top search box. The first result was a 
copy of the original title. It took me about 10 seconds. In comparison, 
2 minutes in the library catalog is an eternity. And a 67% success rate? 
That doesn't sound like a good benchmark to shoot for.

My (very limited) impression of benchmarking in usability testing, 
however, is that benchmarks are used to measure improvement. So you 
might discover how long it takes for users to find a copy of the book in 
the OPAC now and hope that a future usability test shows improvement.

As Karen mentioned, users do more in the catalog than just looking for 
known-items. But it's difficult to measure the duration required to 
complete a task as subjective as finding items about a topic.

-emily

Roger Fenton wrote:
> Do members have any target success rates for various OPAC tasks? I read 
> figures in the literature such as 67% of participants successfully 
> completed the task of finding a copy of The Red Pony in the OPAC in 
> under 2 minutes, but I have yet to find a statement as to whether or not 
> that's considered acceptable or warrants further work on the interface, 
> except by implication when a report states that XYZ was changed as a 
> result of the first round of testing. I have also been unable to find 
> any benchmarking figures of this type.
> 

-- 
Emily Lynema
NCSU Libraries Fellow
Research and Information Services
Information Technology
919-513-8031
emily_lynema at ncsu.edu