NCPA - National Center for Policy Analysis

A Scientific and Medical Quandary

April 24, 2002

At times, scientific studies yield conflicting conclusions -- and the question becomes which to believe.

The issue pits epidemiological or observational studies -- in which large groups are tracked, often for years -- against clinical trials in which participants are randomly assigned to receive, for example, a drug being tested or a placebo. The controversy recently surfaced in research into hormone replacement therapy in women.

  • With hormone replacement, the conclusion from studies that followed hundreds of thousands of women for years was that women who used it were healthier, with less heart disease and longer lives.
  • In contrast, the emerging results of randomized studies are not finding any such effects, and a prestigious group of 28 scientists and doctors, authors of the International Position Paper on Women's Health and Menopause, said hormone replacement therapy's established benefits were much more limited than many doctors and women had believed.

A drawback of observational studies is that people choose their treatment or behavior, and those who choose to take vitamins or exercise are different from those who don't -- thus raising the question whether the result is an effect of the hormones, some other behavior, or a combination of factors.

But randomized studies aren't always accurate either. For instance, small, randomized studies showed vitamin E protected against heart disease -- as did observational studies. Yet a recent large randomized study involving 50,000 people showed vitamin E is not protective.

One explanation is that if the effect is very large -- like the link between smoking and lung cancer -- an observational study will correctly find it. But if there is no effect, or just a tiny one, chance comes into play, and any flaw in a study can throw the results in the wrong column.

Source: Gina Kolata, "In Public Health, Definitive Data Can Be Elusive," New York Times, April 23, 2002.


Browse more articles on Health Issues