NCPA - National Center for Policy Analysis

Tips on Detecting Junk Medical Studies

October 23, 2002

The press and other media have taken it upon themselves to interpret medical studies -- a task which many critics say those institutions are not up to. The result has been a host of scary stories distorting researchers' actual findings. So, the job of understanding the relevance of research has fallen to the general public. Here are some tips:

  • Studies which try to correlate medical results in animals with probable outcomes in humans are particularly suspect -- since metabolism and immune defenses in animals may differ sufficiently to make the finding irrelevant to people.
  • How participants in a study are recruited can influence the reliability of the findings -- for example, if the study is done among people with advanced disease, the conclusions might not apply to people with different or milder forms of that disease.
  • Look at how the study was structured, since the question the study was designed to answer limits the dependability and extendibility of the results.
  • Consider duration, because a valid finding in some studies might require following 22,000 participants over seven years, while another might only require following 6,000 patients over five years or less.

Ask who paid for the study and whether the research findings were independently evaluated. But just because a drug company pays for a study doesn't necessarily mean the research is tainted.

Finally, remember even the most thorough and careful study may require independent confirmation before it is considered a fact that should change medical practice. Rarely does one study bring about a major change in disease treatment or prevention.

Source: Jane E. Brody, "Separating Gold From Junk in Medical Studies," New York Times, October 22, 2002.


Browse more articles on Health Issues