Never knowingly understated
| 21 February, 2011 | Richard P. Grant |
2
|
|
It’s no big secret that we’re not fans of the journal impact factor. So it’s possibly justified to feel a little smug that overstating conclusions of research is positively correlated with impact factor.
Fig 1, Gonon et al.
F1000 Member Noam Ziv evaluates a paper on PLoS ONE that looks at the relationship between misleading conclusions as reported in the media and the misrepresentation or overstatement of the underlying data in the literature itself. It doesn’t make for pretty reading:
we speculate that data misrepresentation in the scientific literature might play a part in the distortion of data into misleading conclusions in the media.
With medical research, this isn’t just an intellectual exercise, of course. The authors look at attention deficit hyperactivity disorder (ADHD), and draw the following conclusion (my emphasis):
Unfortunately, data misrepresentation biases the scientific evidence in favor of the first position stating that ADHD is primarily caused by biological factors. Therefore, this misrepresentation does have social consequences regarding ADHD management: it favors medical interventions over prevention and psycho-social interventions.
This ties in with Sir Iain Chalmers’ attack on the way biomedical research is done. It goes beyond simple non-publication of negative (or ‘disappointing’) results, but into the territory of stretching data to say what they don’t want to say. Having said that, it is perhaps understandable, given the pressures of funding and to publish in as high profile journal as possible, that people will overstate their findings.
Understandable, possibly: but when the data so published have real effects on patients’ lives, then we have to ask whether striving to publish into high impact factor journals is in any way ethical.
I’m not convinced it is.
Gonon, F., Bezard, E., & Boraud, T. (2011). Misrepresentation of Neuroscience Data Might Give Rise to Misleading Conclusions in the Media: The Case of Attention Deficit Hyperactivity Disorder PLoS ONE, 6 (1) DOI: 10.1371/journal.pone.0014618
|
|
previous post
What also may be missing in this discussion is the influence of journals many of which, for prestige and sometimes for profit, have an interest in encouraging the overstatement of conclusions. I once was astonished to find that a journal had altered my title from “xxx suggests” to “xxx shows” in my page proofs (i.e. post-review) . We had them return the title to my preferred more cautious claim prior to publication. It ultimately is the authors who must display the scientific rigor in the face of such pressures to not overstate their findings. Those pressures come from multiple sources for multiple reasons.
Overstatement and misinterpretation of data are outcomes of pressure of so called
“publish or perish” policy.