National Center for Policy Analysis
MONTH IN REVIEW
Government
August, 1996
PEER REVIEW IS NO GUARANTEE
Journalists tend to report the results of research published in academic
and scientific journals as factual or authoritative because it is "peer
reviewed." Peer review means that experts in that particular field
reviewed the methods and conclusions stated in an article and found them
sound. But how reliable is the peer-review process that helps give these
journals such authority? Critics say the answer is "not very."
Actually, the editors of journals decide which articles are even considered
for publication, and may ignore the opinion of reviewers. For example,
- The Journal of the American Medical Association may send an
article submitted for publication to as many as 10 specialists in a field.
- But there are times when only two or three actually participate in
JAMA reviews, and only one expert may end up actually reviewing the
article.
- And there is no rule that the reviewers must unanimously recommend
publication, or even that a majority must recommend publication.
- In contrast, the British medical journal Lancet uses only two
reviewers, but will not publish if both of them review an article negatively.
- The New England Journal of Medicine, which also uses two reviewers,
will print studies panned by both reviewers.
Reviewers are never identified with a particular study, on the assumption
that anonymity secures a higher level of objectivity; and the number of
reviewers or the review process an article has undergone isn't disclosed.
Critics say it is ironic that journals devoted to publishing scientific
research use an unscientific method. Editors may attempt to screen out poor
research, but they are usually generalists in their field. Given the weight
attached to scientific opinion, it is dangerous that the process may depend
on the integrity of individuals.
Source: Neal B. Freeman, "Peer Review and Its Discontents," Weekly
Standard, August 26, 1996.