[Web4lib] Re: Task completion success rates

[John Kupersmith]

In 18 usability tests done at 13 different libraries, the overall average 
success rate for finding journal articles or article databases is 52%.

The median rate is 58%.  Four tests had success rates below 20% (three of 
these were zero), and three tests had rates over 80%.

I've been keeping track of this as part of an ongoing study of library 
terminology, since narrative test reports indicate that terminology plays a 
significant role in user performance.
< http://www.jkup.net/terms.html >

Other factors such as graphic design and information architecture also play 
a part, and it's often difficult to tease out which is most influential in 
a particular case.

There's naturally some variation in specific tasks among the 18 tests, but 
they all cluster around the issue of getting users to find articles.  Given 
the importance of this resource and the amount of money libraries spend on 
article databases, this seems like a worthwhile focus for usability research.

Anyone with test results documenting effective or ineffective terminology, 
or a success rate as described above, is invited to send me the information.

--jk
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   John Kupersmith        jkup at jkup.net        http://www.jkup.net
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   Reference Librarian                 http://www.lib.berkeley.edu
   Doe/Moffitt Libraries
   University of California, Berkeley
~~~~~~~~~~~~~~~Not speaking for UCB in this message~~~~~~~~~~~~~~~~



At 04:20 AM 5/31/2006, you wrote:
>Do members have a target % of successful task completions when doing
>usability tests on an OPAC (or failing that, on a library website)?
>I need to set some objective success rates for upcoming tests. I notice
>that Jakob Nielsen recently quoted 66% as the average success rate (not
>a mark to aim for but the actual success rate) for 'mainstream'
>websites, but he doesn't say whether 'mainstream' means B2C websites or
>also non-commercial sites. Surely we should be aiming for substantially
>better numbers than that?
>
>--
>Roger Fenton
>
>Swyddog Prosiect
>Adran Gwasanaethau Casgliadau
>Is-adran Systemau
>Llyfrgell Genedlaethol Cymru
>Aberystwyth, Ceredigion SY23 3BU
>Cymru
>
>http://www.llgc.org.uk/
>Ffôn: +44 (0) 1970 632800 est. 368
>e-bost: roger.fenton at llgc.org.uk
>
>Dydy'r uchod ddim o reidrwydd yn cynrychioli polisi'r LlGC
>
>Project Officer
>Department of Collection Services
>Systems Section
>National Library of Wales
>Aberystwyth, Ceredigion SY23 3BU
>Wales
>
>http://www.llgc.org.uk/
>Tel.: +44 (0) 1970 632800 ext. 368
>Fax: +44 (0) 1970 632882
>e-mail: roger.fenton at llgc.org.uk
>
>The above does not necessarily represent NLW policy