blog

A systematic review by Tony Ross-Hellauer, University of Gottingen, Germany, tries to resolve the ambiguity surrounding open peer review by providing more than a 120 definitions. It was openly peer reviewed by Bahar Mehmani, Reviewer Experience Lead from Elsevier. In this blog, both explain why there can’t be a single precise definition for open peer review and instead communities should clarify their meaning by naming specific traits, such as open identities, open reports or open platforms, to minimise confusion.

In this review, Tony explains that the definition of open peer review is contested ground. Why is there so much confusion about the definition?

Tony Ross-Hellauer

TRH: The term open peer review was coined more than thirty years ago but only really came to common usage in the mid-1990s. At that time, open peer review usually referred predominantly to opening reviewer identities, in distinction to blinded peer review. But the connectivity of digital networks led others to see that peer review can be opened in other ways, by making reviewer reports public, by enabling mass-commenting on articles, and so on. Peer review is not a monolith and there are any number of ways in which the process can be altered, and as the label “open” has been very fashionable in scholarly communications, I think this explains why many people have gravitated towards using the term open peer review for their favoured systems.

Bahar Mehmani

BM: Different communities have different understanding of the concept. For example, in some communities it is used to define a peer review model in which the identity of reviewer is revealed in the reviewer report, published alongside the article, that is open to readers.

How would the community benefit from a standardized definition?

TRH: Debate about the value of open peer review is often clouded by confusion about exactly what traits are being discussed. Differing open peer review elements need not go together, and they potentially have very different benefits and drawbacks. The impetus for the paper was being at a conference where one researcher put forward the broad point that open peer review was unworkable because if reviewers are known to authors, authors who receive a bad review will hold grudges in future interactions. An extended discussion ensued where others argued back that open peer review has the benefits of enabling scrutiny of the review reports and enabling broader discussion from the crowd.

“The review process is so central to the validation of scholarship that it should not be conducted in the shadows.” Tony Ross-Hellauer

I just felt the conversation was going in circles because the participants were clearly arguing about separate points. The first was discussing the drawbacks of one trait, open identities, which they took to be the sole defining characteristic, but others were countering by listing the benefits of other traits like open reports, open pre-review manuscripts, open participation, rather than the matter at hand.

BM: Knowing what a journal means by practicing open peer review the community around it will know what to expect upon submitting their manuscripts, reading an article and most importantly accepting and writing a review report for an assigned manuscript.

Should we accept that open peer review should be used as an umbrella term? How does this solve the issue and should we think of an alternative definition of peer review?

TRH: I don’t think there is any alternative, the term is used so broadly that it would be difficult to try to reclaim it for any one configuration of these traits. I think that’s fine, as long as we know specifically which traits we’re discussing though.

“The open peer review umbrella term has something to offer to each community depending on their level of openness.” Bahar Mehmani

BM: It seems there is a general agreement on the fact that the term open peer review covers a wide range of variations of standard peer review models. Accepting the overarching nature of the term makes it clearer to whoever is using it to be more specific about the details of the discussed model. I don’t think an alternative definition is able to clarify the nuances. What we miss is a proper taxonomy of relatively new peer review models.

This article was openly peer reviewed. Do you think this was helpful and should this model of peer review become more commonly practiced?

TRH: Definitely, for the various reasons given in the article. I just feel that the review process is so central to the validation of scholarship that it should not be conducted in the shadows. But I don’t think that one-size-fits-all, and I’d really like to take an evidence-based approach to its introduction – what works, in which circumstances, for which communities? My article shows there is very little evidence to answer these questions.

BM: Undoubtedly, transparent peer review fits best with the subject of this specific article. I believe improving the transparency around the peer review process in specific and scientific evaluation in general is helpful for the whole community. Whether the specific way used for this article should become more common is a question though. The good news is open peer review umbrella term has something to offer to each community depending on their level of openness.