blog

Celebrating Peer Review Week 2019 and being a proud member of the industry-wide Peer Review Week committee, has afforded us a welcome opportunity to reflect upon and discuss this year’s theme of ‘quality in the peer review process.’

So, first things first, what do we mean by quality in peer review? Do different stakeholders involved in the process define it differently? To find out, we spoke to reviewers, authors and members of our own team to understand how they define quality in this context and how the F1000Research model of open peer review supports this.

Transparency is the marker of true quality

It’s fair to say that from our numerous conversations, quality in peer review can appear, at first, to be multifaceted. A variety of concepts such as ‘fairness’, ‘honesty’, ‘reliability’ and ‘rigour’ were commonly used throughout our discussions, as highlighted in the quotes below:

“High-quality peer review is critical for science. For me, it means an honest, critical, constructive review of the work, with the aim of improving the science. What it should not be is a gate-keeping exercise that focuses on whether the work is or is not published” Nick Schurch PhD, Biomathematics and Statistics Scotland and Author on F1000Research

“Quality in peer-review is a combination of fairness, rigour and reliability. It is very important to reach as high-quality standards as possible, as the dissemination of scientific discoveries relies upon that, although in reality, a perfect balance of those components is hard to obtain” Marina Lizio, PhD, Laboratory for Genome Information Analysis, RIKEN and Reviewer on F1000Research

“Quality in peer-review is impartial, a thorough evaluation of scholarly research components in a scientific field. Great peer-review examines that the data behind a manuscript are well analyzed and that the presentation of the results and methods allow reproducibility by the fellow researcher” Sarvenaz Sarabipour, PhD, Postdoctoral Fellow, Johns Hopkins University and Reviewer on F1000Research

But what really struck us is that we can only be fair, honest, reliable and constructive if the process of peer review is truly open for all to see. Indeed, one of the things that we are most proud of at F1000Research is our pioneering model of open peer review, whereby each article we publish includes all peer review reports, reviewer names, affiliations and author responses. We believe that transparency is the marker of true quality; reviews of this nature offer numerous advantages to not only the authors but also to reviewers and society.

So, let’s look at the advantages of transparent peer review in turn:

Encourages a constructive dialogue

One aspect of open peer review which is often overlooked is the tone in which it is conducted. While peer review reports should always be constructive and objective, there is anecdotal evidence that peer review reports in closed peer review systems do not always meet these standards. Peer review reports are of great importance to the science community and allow authors to improve on their work, but open peer review also encourages reviewers to praise the methods which have been used. 

Many peer review reports will often include positive statements, such as “The paper is well written and the data used, from the ALSPAC, is of high quality” and others give well thought out constructive feedback, such as “Overall this protocol report is clear and concise, displaying good justification for the research to be conducted, and a strong understanding of the methodological process by which the study aims can be met. I suggest a few amendments which could be made to enhance this paper further.”  

While reviewers are still expected to highlight any significant concerns they have about the work, the fact that their review is fully transparent helps to ensure that reviewers remain professional and are not overly harsh when expressing any concerns. A good, yet informal example of this is this review by Charles Cook.

F1000Research also encourages the practice of co-reviewing. Co-reviewing allows colleagues who may have different expertise to work together, ensuring a comprehensive review.

Open peer review can also lead to a discussion between authors and reviewers via the comment system on F1000Research. Authors and reviewers often respond to one another expressing their thanks to the reviewers.

We have demonstrated that the transparent nature of open peer review gives more constructive and helpful reviews and we continue to find that this provides a platform for discussion between authors and reviewers.

Removes editorial bias

Another element of peer review which we feel has benefited from the process of being open and transparent is that of editorial bias. When authors have made the required revisions, suggested by the closed peer review process, it can still fall to the editor to make a final decision. Many editors care about novelty, as well as how an article fits in with the rest of the content in that edition of the journal. The removal of this editorial bias allows authors to publish what they feel is important and allows the community to decide whether it is going to pass peer review and become indexed.  

Claiming credit

Peer review is a hugely important aspect of science and those outside of the industry are often surprised that there are thousands of people going around behind the scenes, providing peer review reports for free, and receiving little to no credit. If all peer review reports are published alongside reviewer names and affiliations, this gives the reviewers the opportunity to claim full credit for their work and receive recognition for the hard work which goes into their reports. For early-career researchers, this helps them gain experience and provides a boost to their reputation within the research community.

Thanks to sites such as Publons and ORCID, which allow reviewers to add reports to their profiles, peer review contributions are becoming much more visible, and it is now much clearer the amount of effort people put into the peer review process. 

Learning from others

Open peer review can also lead to a discussion between authors and reviewers via the comment system on F1000Research. Authors and reviewers often respond to one another express their thanks to the reviewers.

F1000Research also encourages the practice of co-reviewing. Co-reviewing allows colleagues who may have different expertise to work together, ensuring a comprehensive review. In addition, we encourage PhD students and post-docs to co-review articles, as this can provide valuable experience. This was highlighted by a preprint which can currently be found on bioRxiv, which shows that up to 50% of early-career researchers have ghost-written a peer review report. All co-reviewers, including PhD students, should be named to gain credit for any reviews they contribute to. This is important to F1000Research, as we are keen to ensure that early career researchers get the recognition that they deserve.

Conflicts of interest

Conflicts of interest have always been a grey area in scientific publishing. Using the more traditional closed model of peer review, readers would not have known whether editors and peer reviewers were influenced by personal or financial relationships with other people or organizations, which may have provided a bias for their opinion on the article.  

In more traditional peer review systems, it is not possible for readers to check for themselves that the reviewers had no conflict of interest when reviewing the article. To ensure that everything is fully transparent, at F1000Research we ask that all authors and reviewers declare any potential conflicts of interest, and these are published alongside the respective article or review. We also ask that commenters on the article state whether they might have a competing interest. Readers are then able to decide for themselves whether these may have influenced the content.

The future is open

And whilst we are not too proud to admit that our open peer review model has been met with some scepticism in the past, we have been heartily encouraged by the growing movement and acceptance towards this truly open method of publishing.

Recent articles and studies from The Publication Plan and Nature, showcase the ever-increasing success and desire amongst researchers to embrace this new publishing process, as does a Twitter poll this week run by the Peer Review Week industry campaign which resulted in 74% of people saying that they believe transparent peer review can improve the quality of reviews. An attitude which has certainly been echoed by our own authors, as highlighted in the results of our recent author survey:

As open science continues to grow and become a normal part of science research, we feel that open peer review will also be part of this movement, encouraging greater transparency in science. 

If you’re interested in learning more about open peer review, volunteering as a reviewer or how you can publish with us, please visit F1000Research.