blog

Earlier this week I attended (and spoke at) a workshop about publication bias, organised by NC3Rs – the National Centre for the Replacement, Refinement & Reduction of Animals in Research. This group is interested in the publication bias issue because using animals in research adds an extra responsibility to publish all of the results of that work. (See also this guest post by ZonMW we previously published.)

In the opening lecture of the meeting, Emily Sena, of the University of Edinburgh, introduced the evidence and impact of publication bias. A lot of what she showed us is in her 2010 PLOS Biology article, which outlines the methods she used to identify publication bias in stroke literature.

After this introduction, the rest of workshop was focused on three different topics, and discussions about whether they would be able to address publication bias: prospective registration of preclinical studies, publishing and data sharing, and publication bias in industry and toxicology.

 

Prospective registration of clinical trials

The first topic, prospective registration of preclinical trials, explored whether animal research should adopt the model now starting to be used in clinical research, where clinical trials are registered with a publisher before they are carried out, to improve the chances of the researchers publishing the finished study, regardless of outcome. Jonathan Kimmelman (McGill) started by outlining some of the problems caused by not publishing the results of all preclinical studies. Most dramatically, patients can be harmed by insufficient records of preclinical studies. He advocated emulating the existing model for registration of clinical studies, which tied in nicely to the next talk, by Trish Groves (BMJ). She outlined the history of clinical trial registration, from the Helsinki Declaration in 1964 until the recent AllTrials campaign. Since 2008, all ICJME-affiliated journals require pre-registration of clinical trials before the outcome of the trial is published.

So how can this be done for preclinical studies? Chris Chambers (Cardiff University) described how Cortex accepts “registered reports” of experimental work, which, when accepted by peer reviewers, guarantees “in principle acception” of the final manuscript. After an open letter in the Guardian, they also convinced other publishers to try a similar system.

 

Debate

This topic, and the first day of the meeting, was closed with a debate, pitting the three speakers against an assigned opposition team. Before the debate, the vast majority of the room agreed with the statement “This house believes that prospective registration of preclinical studies is necessary to reduce publication bias and its impacts”. After the debate, in which the opposition based their arguments on the large amount of work that preclinical registration would involve, and the limitations it would pose on the meandering creativity of researchers, a big part of the audience had now changed their mind and was more skeptical of the ability of preclinical study registration to reduce publication bias. (For the record: my own vote remained in favour of registration. The arguments of the opposition only pointed out that it would be a difficult process.)

 

Publishing and data sharing

The majority of day 2 of the workshop was focused on data sharing and new publication models. Talks from Mark Hahnel (Figshare), Susanna-Assunta Sansone (Scientific Data), Damian Pattinson (PLOS One), me (F1000Research), and Christophe Bernard (eNeuro) introduced some solutions our respective organisations had found to address the problem of publication bias. Readers of this blog will be familiar with most of these, but you might not yet have seen PLOS One’s new “Missing Pieces” collection, launched this week. My own talk focused on F1000Research’s negative results campaign, and the features around living articles and living figures.

Closing this session, after lunch, was Tom Walley (NIHR), who gave a funder’s perspective on the need to share all research data. He showed how they managed to get a high compliance rate (98%) for publication of trials in particular by providing a funder-run journal. If they only considered trials published in other journals, compliance was as low as in other countries (around 50%).

 

Publication bias in pharmaceutical industry and toxicology

The last session was a trio of talks describing issues specific to industry and toxicology research, by Glenn Begley (Tetralogic Pharmaceuticals) Noel Dybdal (Genentech) and Colin Berry (QMUL). Overall, industry experiences many of the same problems as academic researchers when it comes to publication bias, but Dybdal did point out one unigue problem for industry, where publishing negative results may actually lead to a stock drop!

 

Group discussions

The day ended with discussions in breakout groups, around the topics of prospective study registration, publishing models, and data repositories. Of these, data respoitories were the least controversial, with everyone agreeing about their benefits, and identifying barriers to compliance.

 

The organisers will publish a report about the full workshop that should have a lot more information, so keep an eye out for that.