blog

Earlier this month I had the chance to be on the panel of one of Sense About Science’s peer review workshops. Sense About Science is an organisation that helps people make sense of scientific information. For example, they run campaigns to encourage people to ask for evidence when beauty products make scientific claims, but they also promote the value of peer review, and this year they organised three workshops through the Voice of Young Science network to introduce early-career researchers to peer review and also address some of its pitfalls.

With me on the panel were Professor Sheila Bird (MRC Biostatistics Unit at the University of Cambridge), Chris Greenwell (Publishing Director, Mathematics and Statistics at Elsevier), and Victoria Murphy of Sense About Science.

Sheila talked about her experiences as a journal editor, as reviewer, and as author, which really brought home the fact that these roles are not carried out by different groups of people, but that one scientist can be involved in many different aspects of peer review.  I talked about different types of peer review, from the traditional single-blind prepublication review to the open post-publication review we use at F1000Research, and all things in between. Chris closed the introductory talks with a bit of a historical broad overview of peer review in general.

In the discussion and group assignment section of the workshop we heard some of the participants’ views about peer review. They found peer review to be useful as a form of quality control, to get feedback on their work, and also to improve their papers, but they also thought it was often very slow, and it could be inconsistent.

Statistics reviews

The workshop was held as part of the Royal Statistical Society conference, so a lot of participants were statisticians, and I learned that statisticians have a slightly different experience with peer review than most other scientists. They are often called upon to review the statistics used in papers from other fields. Sometimes they will even be paid to do this. One of the audience members, who had experience being the statistical reviewer, said that he found it wasn’t always fair to weigh reviews equally: If he finds grave statistical errors in the data analysis in a paper, that shouldn’t be overruled by other reviewers recommending it for the interesting data and unique angle.

Ethics

Another topic that came up during the workshop was publication and refereeing ethics. Emily Jesper of Sense About Science told the attendees about COPE and its role in overseeing publishing ethics as well as serving as a central point for editors to go to with individual issues that arise at their respective journals. (F1000Research is a member of COPE as well.)

One ethics issue that we talked about in particular was the impartiality of peer reviewers. Most journals will make an effort to ensure that reviewers do not have a stake in the publication of the paper, but occasionally things slip through the cracks. For example, Chris mentioned a case where reviewers managed to very subtly insist that their own papers were included as references in papers they reviewed.

I mentioned how open peer review can help catch some of these things. For example, commenters may know of conflicts of interest between referees and authors that weren’t picked up, and can offer more context.

Education

Finally, we talked a bit about learning how to do peer review. One audience member said that she started out by doing a joint review with her supervisor, and this seems to be a common way for people to take their first steps in becoming a referee.  Others mentioned that they took a cue from reviews they received on their own papers .That can work if you’ve already had several things reviewed and have some good examples.

I showed a group I recently set up on Mendeley, where people can post papers with exemplary open peer reviews, as a resource for examples of referee reports. There are currently a few examples in there, mostly from F1000Research (because I set it up…) and a few from PeerJ. As more and more journals are now using open peer review, it’s becoming quite easy for young scientists to find examples of peer review done by others.

Sense About Science has also published a guide about peer review, which is free to download from their site. In the preparation for my own talk I used a similar guide produced by the British Ecological Society, which is also available online [pdf]. Both guides are very good and up-to-date resources about peer review for young scientists.

 

(Images used with permission from Sense About Science. See more on the Voice of Young Science’s Facebook page. )