ARE SCIENTIFIC PAPERS RELIABLE?
September 13, 2005
Scientific papers are the primary source of evidence available to doctors and lawmakers in forming their decisions. However, an article from PLoS Medicine (an online journal) finds that majority of these papers reach false conclusions.
The author analyzed 49 research articles printed in widely read medical journals between 1990 and 2003 that were cited by other scientists 1,000 times or more. Some of them were later proven incorrect. He finds that several things can distort the accuracy of such reports, such as small sample sizes, studies that show weak effects, researcher bias, and poorly designed experiments.
Next, the author designed a mathematical model to predict the accuracy of studies in general. He found:
- Even a large, well designed study with little research bias has only an 85 percent chance of being right.
- An underpowered, poorly performed drug trial with research bias has only a 17 percent chance of producing true conclusions.
- Overall, more than half of all published research is probably wrong.
The author does note that medical science is not the be-all and end-all of research. The physical sciences, with more certain theoretical foundations and well-defined methods, probably do better than medicine.
Source: "...and statistics," Economist, September 3, 2005; based upon: John P. A. Ioannidis, "Why Most Published Research Findings Are False," PloS Medicine, Volume 2, Issue 8, August 2005.
For text (subscription required):
Browse more articles on Health Issues