blog

It was Kuhn, quoting Max Planck, who claimed that for science to progress (through the discarding of old hypotheses and the adoption of new ones) that opponents of progress had to die. It remains to be seen whether the general acceptance of that hypothesis itself will outlive the current generation (or not). But as our friends at Retraction Watch might want to remind us, it is undeniable that old, incorrect papers continue to be cited.

Is it a problem?

A paper looking at citations of rebutted paper, and perhaps more importantly the attitude (from rejection to uncritical acceptance) of those citations, appears to suggest that it is. (For anyone just joining us, I should point out scientific findings are invariably rebutted. It’s how science progresses: we think we know something; we publish it; someone else–usually–discovers that the original finding isn’t correct, or is incomplete; we progress in our understanding and the boundaries of ignorance retreat infinitesimally.) Jeannette A. Banobi and colleagues at the University of Washington studied seven widely cited (and subsequently rebutted) papers in the field of fisheries ecology and management. They found that the seven papers were cited a total of 2982 times, and their 24 rebuttals 323 times: “the original articles averaged 17 times more citations per year than the rebuttals”^{10.1890/ES10-00142.1}.

But the real surprise is that a rebuttal didn’t decrease the citation rate of the original article–and the apparent regard for the original paper wasn’t adversely affected either. (Interestingly, rebuttals had a higher chance of being cited if they were published in journals other than Science or Nature. This is probably because those two journals are where you read about sexy, trendy research, whereas other journals are where you find the real work being done.)

I’ll let the authors summarize for you, in their own words:

Our results provide strong evidence that rebuttals scarcely alter scientific perceptions about the original papers.

The authors call for editorial measures to combat this effect. Presumably these might take a similar form to how some journals link retracted papers forward to the retraction notice. We already have mechanisms in place to find out what papers cite an article (and Google Scholar should make this easier); it would take a little bit more effort to mark rebutting citations as such on an online article. Would that be enough? Please read F1000 Member Ferdinando Boero’s entertaining evaluation, and then let me know what you think we should do to make publishing more joined-up.