website usability testing administration questions

[Laura Hudson]

Dear Colleagues,

We're beginning some website usability testing (card sorting tests and then 
timing of question answering) here at the San Diego State University 
library in order to determine how to best arrange our website.  While I've 
been managing large websites for years, I haven't done this before, so I 
have of course read a bunch on the topic.  However, in our webteam 
meetings, questions such as the below come up and I don't have the answers:

* When doing card sorting tests, how many content objects can we expect our 
subjects to be able to handle and what level of granularity is appropriate?

* Are subjects, when card sorting, coming up with totally bizarre 
organizations, or are they generally what you'd expect, or somewhere in 
between?  Does anybody have examples of how they've arranged something 
differently due to card sorting tests?

* Because library websites are diverse-audience sites, do you always test 
with all the potential groups of people (e.g. undergraduate students, 
graduate students, faculty, staff, librarians), or do you pick your main 
audience and run with that?

* How are you coping when you don't have a budget (can't pay subjects or 
would have to hold a bake sale to do so) and are tightly staffed (don't 
have a lot of time or staff support for this)?  Or are most of you 
finagling budgets and finding time to slough off some of your other 
committees, meetings, and projects for the duration?

* When conducting the actual usability tests, how many wire models do you 
construct, or do you start with one, improve, and then test again until you 
are happy with the results?

* Do you videotape usability tests? If so, how do you analyze the 
tapes?  Did you find doing so useful?  Necessary?  Or do you think that the 
quantitative data (e.g. how long does it take them to answer the question) 
is more important?

I know that a lot of you are getting into this fascinating area of 
research.  If any of you have a) answers to any or all of these questions 
b) war stories c) favorite resources and/or d) citations to your own web 
redesign process websites or articles, I would be terribly grateful.  I 
would also be delighted to compile answers for the list.

Thank you in advance!

Laura Hudson
Electronic Systems and Services Librarian
San Diego State University
lhudson at mail.sdsu.edu
phone: 619-594-3521
fax:  619-594-8982