[WEB4LIB] RE: Online peer-review publishing and post-peer rev iew

[Dobbs, Aaron]

A very good question with a complex answer.  (that I might not sufficiently
answer here)

Say you are a new student or not very familiar with the subject area of that
which has been published, you read the article and it makes enough sense to
you that you accept the author's thesis. You then proceed to the post-peer
review section and the first review/discussion of the article says the
author is a fraud and perhaps provides seemingly irrefutable proof to that
effect.  Now what?  Do you merely ignore the article and assume your
acceptance of the thesis was wrong?

To make a contrived argument as an example:
The author's thesis is stated thusly: prevailing wisdom says gravity affects
different weights differently; if I drop a 1 ounce feather and a 5 pound
rock from the same height the rock will hit the ground first. However I
think this is not so; I have tested this several times with a 1 ounce weight
and a 5 pound weight and both hit the ground at the same time, please see
attached video for confirmation of my thesis. [attached video does, in fact,
show both items dropped from same height and both landing at the same time].
The first review/discussion states: the author is a misguided crank. He used
differently sized containers of the same weight to achieve his results.
Plus I'm a member of [pick a group that people implicitly trust] that most
people recognize as honest, so you know I'm right.
So, how does a person not steeped in the discipline know who to believe?  We
all probably have our own opinion about the example above and were we all to
"moderate" the discussion that would follow a possible consensus could be
derived.  Most likely several other moderators would replicate the
experiment to attempt to disprove or support the thesis and would then post
their results and add to the discussion, which could mitigate the "first
post" conundrum.  These follow on replications would, in a perfect world, be
assigned more significance than that of the first poster who merely says
"that ain't so" and provides no evidence to support the claim.

The moderation would be more to assign significance to in-depth discussion
and to marginalize non-constructive commentary.  Assuming a large enough
body of moderators that are active in the discipline (checked upon by a
group of meta-moderators - who are also members of the moderators group but
not allowed to meta-moderate discussions in which they are active) the more
egregious "moderations" will be indicated as such.  Slashdot.org's
moderation system is, loosely, what I was thinking of when mentioning
moderation originally.  

-Aaron
:-)'

His philosophy was a mixture of three famous schools -- the Cynics, the
Stoics and the Epicureans -- and summed up all three of them in his famous
phrase, "You can't trust any bugger further than you can throw him, and
there's nothing you can do about it, so let's have a drink."
--Small Gods, Terry Pratchett



-----Original Message-----
From: latham1 [mailto:latham1 at students.uiuc.edu]
Sent: Wednesday, January 15, 2003 9:46 AM
To: DobbsA at apsu.edu; Multiple recipients of list
Subject: RE: [WEB4LIB] RE: Online peer-review publishing and post-peer
review


Why would moderation be required?  I know academia can get a bit testy, but
in 
this instance a moderator could only function as a censor ... and the whole 
issue in academia is to allow the range of ideas ...

Joyce Latham
GSLIS

>===== Original Message From DobbsA at apsu.edu =====
>While I like the modification of the tenure process "...the goal with
>post-peer review publishing would be to reach consensus on a stated problem
>or issue and reward, not just the author(s), but also the reviewers who can
>act as content contributors." idea, I feel that reaching consensus on a
>given issue may perhaps not neccesarily be the goal of the research
process.
>Research seems more a wide ranging discussion of the issues at hand with
all
>theories debatable and open for honest consideration, even the untenable
>pie-in-the-sky-not-a-chance-in-heck cranks can occasionally spark an idea.
>
>For the discussion board type approach (which seems to be what is
described)
>to work there would likely need to be a moderation system and a
>meta-moderation system (and perhaps a meta-meta-moderation system) and
would
>also require a non-self-serving ethos among the participants.  And, of
>course, who would moderate the meta- (and the meta-meta-) moderators? :)
>
>-Aaron
>:-)'
>
>Ash nazg durbatuluk, ash nazg gimbatul, ash nazg thrakatuluk agh
burzum-ishi
>krimpatul.
>
>
>
>-----Original Message-----
>From: D.H. Mattison [mailto:dmattison at shaw.ca]
>Sent: Wednesday, January 15, 2003 12:42 AM
>To: Multiple recipients of list
>Subject: [WEB4LIB] Online peer-review publishing and post-peer review
>publishing
>
>
>It's not a problem with online peer-reviewed publications because they
>don't have the hardcopy timelag (typesetting/layout, printing, binding,
>mailing). You could also do post-peer review publishing with a wiki-type
>system where continuous, open feedback is the norm. Of course this doesn't
>quite mesh with aspects of the tenure system as I understand it (publish or
>perish), but given that there're movements afoot to restructure the entire
>scholarly publishing system, what's wrong with post-peer review publishing
>as a process or additional type of scholarly publishing? If you forget or
>modify the tenure system to accommodate such a process, the goal with
>post-peer review publishing would be to reach consensus on a stated problem
>or issue and reward, not just the author(s), but also the reviewers who can
>act as content contributors.
>
>Well, ok, Rome wasn't built in a day ....
>
>David Mattison
>dmattison at shaw.ca

J.M. Latham
Graduate School of Library and Information Science
University of Illinois -- Urbana Champaign
latham1 at students.uiuc.edu