Pumped up findings: Do publishing practices distort science?

By Sarah Arnquist

The credibility of scientific journals and the findings has come under attack multiple times this month via accusations of unethical editing and artificially pumped up findings.

PloS Medicine -- an open-access journal -- published an essay suggesting that current publication methods depict a false portrayal of scientific progress through pumped up results.

The article by an NIH researcher, Greek epidemiologist at Tufts University and a George Mason University economist that suggests current publication practices may distort science. In their review, the researchers view scientific information as an economic commodity and the peer-reviewed journals as the medium of exchange.

"Idealists may be offended that research be compared to widgets, but realists will acknowledge that journals generate revenue; publications are critical in drug development and marketing and to attract venture capital; and publishing defines successful scientific careers."

In describing the article, the Economist wrote:

"With so many scientific papers chasing so few pages in the most prestigious journals, the winners could be the ones most likely to oversell themselves—to trumpet dramatic or important results that later turn out to be false. This would produce a distorted picture of scientific knowledge, with less dramatic (but more accurate) results either relegated to obscure journals or left unpublished."

The result, according to the authors, is publication bias that distorts science -- possibly at the commercial gain of someone or some corporation. The authors suggest their is a moral imperative "to reconsider how scientific data are judged and disseminated."

That brings me to a very interesting discussion that occurred at Wednesday's Health 2.0 panel about the ability of online patient networks to aggregate clinical data and effectively learn and research in real-time.

Ben Haywood, CEO of Patients Like Me, gave an example of how ALS users of the site collected and recorded for a year whether taking lithium slowed the progress of the disease. Their pooled date showed it didn't -- in contrast to a published trial.

Granted, this is not a gold-standard clinical trial but it is useful observational data that can inform science and medicine and hopefully at a faster pace.

There is a need for quality peer-review, but there is also unlimited space on the Web to publish. Shouldn't it be getting easier to publish well-designed studies that show no effect and therefore minimize publishing bias?