NCPA - National Center for Policy Analysis

Measuring In A Subjective Science

January 3, 2001

Many psychological studies may be compromised by an error having to do with the difficulty of measuring patients' subjective responses. So cautions Linda Bartoshuk, a psychologist at the Yale University School of Medicine.

The problem creeps in when researchers use a common measurement technique, the rating scale, to compare different people's subjective experiences -- creating distorted data. For example, two patients may be asked to rate the degree of their depression on a scale of one to 10. Both may put the figure at a 6. But how is the researcher to know if they actually feel the same depth of pain?

Put another way, one man's pain may be another's twinge.

  • After reviewing eight volumes of the journal Psychology and Behavior, Bartoshuk found the error in 17 of 64 human studies that used rating scales.
  • In some cases the mistake completely invalidated the studies' findings, she says.
  • Ratings scales are highly effective when researchers want to find out how the same subjects' feelings, attitudes or sensations change over time or in different situations.
  • Adjectives also mean different things to different people, she points out.

For example, a famine victim's "very hungry" probably indicates a level of hunger far higher than the "very hungry" of a Fortune 500 CEO questioned an hour before dinner.

The mistake is common in other fields as well. Monica Biernat of the University of Kansas and Melvin Manus of the University of Michigan have discovered the same problem in studies of sensory perception.

They discovered that subjects do not always mean the same thing when they apply descriptions such as "tall," "aggressive" or "financially successful" to others.

Source: Erica Goode, "Researcher Challenges a Host of Psychological Studies," Science Times, New York Times, January 2, 2001.


Browse more articles on Tax and Spending Issues