[Web4lib] Library Automation Survey Results

Thu Jan 10 14:43:07 EST 2008

Okay, I'm rethinking what I said earlier, thanks to some feedback from
Bill Drew and some experimentation. I think my mathematical explanation
was wrong, but I do have a fundamental objection to these that assign an
arbitrary quantitative value to purly qualitative states. You can't take
an average of "poor", "fair", "good", and "excellent" by defining
corralating values of 1, 2, 3, and 4, unless you mean to say that "fair"
is twice as good as "fair", but "excellent" is only 1.333 times as good
as "good". But perhaps you can in fact say 0 is "poor" and 9 is
"excellent", without assigning any other qualitative properties in
between, and that is okay then to average the results.

Anyhow, my apologies to the original poster for not thinking more
carefully before lodging my original criticism. I do stand by my
assertion with the type of question / analysis described above, but
clearly I now see a difference between the two types of scale questions,
and the appropriateness for the type of analysis that was used, being
used for each type of question.

- David

---
David Cloutman <dcloutman at co.marin.ca.us>
Electronic Services Librarian
Marin County Free Library 

-----Original Message-----
From: web4lib-bounces at webjunction.org
[mailto:web4lib-bounces at webjunction.org] On Behalf Of Cloutman, David
Sent: Thursday, January 10, 2008 10:14 AM
To: Breeding, Marshall; Web4lib at webjunction.org
Subject: RE: [Web4lib] Library Automation Survey Results

I'm glad to see that you conducted this survey. I think there's a lot of
vendor pressure put out by the providers of ILS systems, and it is nice
to see how they compare. I would however question the validity of how
you approached your statistical anlysis. Averaging the results of an
instrument like this is not a proper way of analyzing the data, and
although I don't remember much of my college statistics class, I do
remember the explanation of why this doesn't work, even though this type
of analysis is done all the time.

In your survey, the difference between a 0 and a 1 means that you are
100% more positive about the question. The difference between a 1 and 2
is 50%, 2 and 3 is %33.3, 3 and 4 is 25%, 5 and 6 16.6%, and so on. That
mean that choosing between a 2 or a three has a much greater impact than
choosing between an 8 or a 9, which skews your results toward the
negative, and diminishes the contrast between positive votes.
Individuals filling out your survery will generally be unaware of this
and will pervcieve the curve of their responses to be linear rather than
curved, with the slope of the beginning curve being much greater than
the slope of the end. 

I don't mean to pick on you, but this is a pet peve of mine. I see this
type of statistical analysis done all the time, including job
evaluations, student / teacher evaluations, and in other criticals
contexts, an it scares me to death, because from a fundamental
statistical standpoint, it can be demonstrated that this is not an
appropriate form of analysis, and typically this is covered in an
introductory statistics course. Still, it is done all the time, and
critical decisions are made from this misleading methodology of number
crunching. I would urge everyone reading this to take a serious look at
the instruments and the analytical methods they use in their
organizations, and to seriously think about this problem.

- David

---
David Cloutman <dcloutman at co.marin.ca.us>
Electronic Services Librarian
Marin County Free Library 

-----Original Message-----
From: web4lib-bounces at webjunction.org
[mailto:web4lib-bounces at webjunction.org] On Behalf Of Breeding, Marshall
Sent: Thursday, January 10, 2008 7:48 AM
To: Web4lib at webjunction.org
Subject: [Web4lib] Library Automation Survey Results

Last August I sent a message to this list soliciting responses to a
survey that I was conducting on libraries perceptions of their library
automation systems, the companies that provide them, and on attitudes
toward open source ILS.

I have closed that survey and have written an article describing its
findings.

The article titled "Perceptions 2007: an International Survey of Library
Automation" is available here:

  http://www.librarytechnology.org/perceptions2007.pl

Thanks to the 1,779 individuals who participated in the survey.

-marshall

------------------------------------------------

Marshall Breeding

Director for Innovative Technology and Research

Vanderbilt University Library

615-343-6094

http://www.librarytechnology.org

_______________________________________________
Web4lib mailing list
Web4lib at webjunction.org
http://lists.webjunction.org/web4lib/

Email Disclaimer: http://www.co.marin.ca.us/nav/misc/EmailDisclaimer.cfm

_______________________________________________
Web4lib mailing list
Web4lib at webjunction.org
http://lists.webjunction.org/web4lib/