[Web4lib] Library Automation Survey Results

[Cloutman, David]

I'm glad to see that you conducted this survey. I think there's a lot of
vendor pressure put out by the providers of ILS systems, and it is nice
to see how they compare. I would however question the validity of how
you approached your statistical anlysis. Averaging the results of an
instrument like this is not a proper way of analyzing the data, and
although I don't remember much of my college statistics class, I do
remember the explanation of why this doesn't work, even though this type
of analysis is done all the time.

In your survey, the difference between a 0 and a 1 means that you are
100% more positive about the question. The difference between a 1 and 2
is 50%, 2 and 3 is %33.3, 3 and 4 is 25%, 5 and 6 16.6%, and so on. That
mean that choosing between a 2 or a three has a much greater impact than
choosing between an 8 or a 9, which skews your results toward the
negative, and diminishes the contrast between positive votes.
Individuals filling out your survery will generally be unaware of this
and will pervcieve the curve of their responses to be linear rather than
curved, with the slope of the beginning curve being much greater than
the slope of the end. 

I don't mean to pick on you, but this is a pet peve of mine. I see this
type of statistical analysis done all the time, including job
evaluations, student / teacher evaluations, and in other criticals
contexts, an it scares me to death, because from a fundamental
statistical standpoint, it can be demonstrated that this is not an
appropriate form of analysis, and typically this is covered in an
introductory statistics course. Still, it is done all the time, and
critical decisions are made from this misleading methodology of number
crunching. I would urge everyone reading this to take a serious look at
the instruments and the analytical methods they use in their
organizations, and to seriously think about this problem.


- David


---
David Cloutman <dcloutman at co.marin.ca.us>
Electronic Services Librarian
Marin County Free Library 

-----Original Message-----
From: web4lib-bounces at webjunction.org
[mailto:web4lib-bounces at webjunction.org] On Behalf Of Breeding, Marshall
Sent: Thursday, January 10, 2008 7:48 AM
To: Web4lib at webjunction.org
Subject: [Web4lib] Library Automation Survey Results


Last August I sent a message to this list soliciting responses to a
survey that I was conducting on libraries perceptions of their library
automation systems, the companies that provide them, and on attitudes
toward open source ILS.

 

I have closed that survey and have written an article describing its
findings.

 

The article titled "Perceptions 2007: an International Survey of Library
Automation" is available here:

  http://www.librarytechnology.org/perceptions2007.pl

 

Thanks to the 1,779 individuals who participated in the survey.

 

-marshall

 

------------------------------------------------

Marshall Breeding

Director for Innovative Technology and Research

Vanderbilt University Library

615-343-6094

http://www.librarytechnology.org

 

 

_______________________________________________
Web4lib mailing list
Web4lib at webjunction.org
http://lists.webjunction.org/web4lib/

Email Disclaimer: http://www.co.marin.ca.us/nav/misc/EmailDisclaimer.cfm