[Web4lib] Library Automation Survey Results

[Cloutman, David]

Okay. Thanks for the explanation. I guess where my skepticism lies is in
how meaningful taking an average of an ordinal scale actually is, when
regardless of the type of scale you are using, the results that you
average are a set. It seems like the set of results from an ordinal
scale are treated by most people as if they were no different than the
set of results from a ratio scale, and to me these seem like
fundamentally different things. 

If you have a set:

{1, 3, 3, 4}

You can say 2.75 is the average. If those numbers represent the ages of
children at a library event that's fine to average, but if we use the
ordinal scale I originally defined, do you think it's fair to say for
the set

{"poor", "good", "good", "excellent"}

the average of ordinal values really means "almost good", because that
is how I would attempt qualify 2.75? If you're looking at a set of
evaluations for a program at your library, for instance, by averaging
the ordinal values, you're taking three positive responses an one
negative response, and deriving a conclusion that the program wasn't
"good", even though 3/4 of the respondents would have said otherwise. To
me that doesn't sound like a fair mechanism for evaluation, particularly
if you have a manager that expects programs to produce a 3, because that
is "good". 

Anyhow, I won't beat this dead horse any further, since I only vaugely
know what I'm talking about and this is getting off topic, but I'm
curious to know what your thoughts are.

- David


---
David Cloutman <dcloutman at co.marin.ca.us>
Electronic Services Librarian
Marin County Free Library 

-----Original Message-----
From: Dan Lester [mailto:dan at riverofdata.com] 
Sent: Thursday, January 10, 2008 12:48 PM
To: Cloutman, David
Cc: web4lib at webjunction.org
Subject: Re[2]: [Web4lib] Library Automation Survey Results


The text below is still incorrect, except for your personal objection
to an ordinal scale. See below.

Thursday, January 10, 2008, 12:43:07 PM, you wrote:

> I think my mathematical explanation
> was wrong, but I do have a fundamental objection to these that assign
an
> arbitrary quantitative value to purly qualitative states. You can't
take
> an average of "poor", "fair", "good", and "excellent" by defining
> corralating values of 1, 2, 3, and 4, unless you mean to say that
"fair"
> is twice as good as "fair", but "excellent" is only 1.333 times as
good
> as "good". But perhaps you can in fact say 0 is "poor" and 9 is
> "excellent", without assigning any other qualitative properties in
> between, and that is okay then to average the results.

There are four types of scales that can be used:

Nominal  (as indicated by the name, this is for things like M/F,
dog/cat, etc. that are categorized by name)

Ordinal:  This is the type used in the survey under consideration.
Things are ranked in ORDER, so a 3.5 is halfway between a 3 and a 4.
A 4 is "one step" higher than a 3, not 1.33 times "better".  We're all
familiar with this from school grades where, say, an A is a 4, a B is
a 3, and so forth.  That does NOT mean that an A is 1.33 times as good
as a B is 1.5 times better than a C.  In fact, if you go by the other
form of ordinal scale that the prof uses, an A may represent 95, a B
85, and a C 75, where the ratio is different.  It is just an ordering,
and NOT a ratio.

Interval: Not relevant here, but works for things like temperatures.

Ratio:  These are where there is a true zero, and where a ratio is
valid, such as age, weight, height, and many other measurements.  If a
building is ten feet tall it is indeed half as tall as one that is 20
feet tall, and that half as tall as one 40, and so forth. (I was going
to use weights, but someone might have some personal problem with
that)

Anyway, you're looking at the ratings as ratio measurements, when
they're really ordinal measurements.

For another source on this, see:
http://www.math.sfu.ca/~cschwarz/Stat-301/Handouts/node5.html or any
statistics textbook.

dan (permanent ABD in statistics and research methodology)


-- 
The road goes on forever and the party never ends. REK, Jr. 
Dan Lester, Boise, ID  



Email Disclaimer: http://www.co.marin.ca.us/nav/misc/EmailDisclaimer.cfm