blog

Well, what a week it has been! It is fair to say that each and every corner of the scholarly publishing world has been abuzz with questions, conversations, and debates, helping to bring a new level of transparency to the state of peer review in 2018.

Here at F1000, we pride ourselves on openness and transparency, and are dedicated to serving authors and their associated research communities. That is why we want research to be immediately available and accessible, and adopt a transparent model of peer review, publishing all reviews along with the name and affiliation of the reviewers.

But we know that in order to be truly open and transparent, our peer review model has to be rigorous and fair. And in order to be both of these, we need a diverse and inclusive pool of reviewers. That’s why we took the opportunity to truly understand just how diverse and inclusive our F1000Research peer review model really is, so that we can actively recognise what we are doing right and actively address any areas of weakness.

To do this, we investigated the following factors: how many co-referees reviewed articles published on F1000Research, our gender balance among F1000Research published reviewers, and the demographic spread of F1000Reasearch reviewers across the world.

Below is a summary of our findings and what we pledge to do to make the F1000Research peer review model much more diverse and inclusive. You can also view the evaluation data in its entirety here.

Giving early career researchers the credit they deserve

Our evaluation found that one fifth of F1000Research articles have at least one co-referee, with this number rising year on year since we launched the co-referee option in 2012. We launched this inclusive opportunity to help early career researchers not only feel more comfortable about reviewing with an open identity, but also to ensure that they are credited for their hard work by being cited in the article itself, as well as on ORCID. It can also be argued that that by including reviewers of different levels of seniority, it will automatically improve other aspects of diversity such as diversity of thought, perspective and experience.

The regional diversity of F1000Research reviewers

Being a global company, we strive to give a global view, to be truly representative of the research community worldwide. Our evaluation showed that we currently have reviewers representing 104 countries, that’s just over half of all countries worldwide, from all continents, except Antarctica. And whilst this certainly makes for encouraging reading, we know we still need to do more to diversify global distribution and move towards a more regionally-balanced reviewer pool.

We know that a multitude of factors can account for regional bias, including personal connections, English as a first language, and the editors’ and reviewers’ own preferences. Whatever the reasons, it is clear to see that reviewers from emerging regions are not being fully included in the peer review process.

Gender bias

According to our research, men make up 74% of published reviewers on F1000Research compared with just 26% females, showing that women are underrepresented in our peer review process. This is simply not good enough.

The American Geophysical Union (AGU) published a study of gender bias — at all ages — in peer review. It concluded that ‘women of all ages have fewer opportunities to take part in peer review…women were used less as reviewers than expected…The bias is a result of authors and editors, especially male ones, suggesting women as reviewers less often, and a slightly higher decline rate among women in each age group when asked’.

Lessons Learned

And whilst it is important to highlight the successes and barriers to diversity and inclusion in our peer review model, we also feel that it is vital that we identify and implement tangible solutions to ensure that we start to make real progress in this area.

Below, we have outlined the steps we will be taking towards addressing these issues:

To be more explicit about diversity and inclusion whilst interacting with our authors

In our finding referees guidelines we suggest that authors should consider selecting reviewers from across the globe to gain an international perspective for their article. Currently, this is the only prompt we have to help authors to consider the regional diversity of the reviewers they select which we realise is not comprehensive enough.

What we pledge to do moving forward is to be much more overt in providing information for our reviewers on the importance of diversity and inclusion when selecting reviewers. We will be identifying the parts of our publication workflow where we can get a clear, explicit message across to the authors when they are selecting their reviewers to consider diversity and inclusion. This will include new text in our reviewer guidelines, new text in our correspondence with authors and also clear information on our “My referee” pages where authors are at the point of selecting their reviewers.

This active approach of appealing to authors to be mindful has already proven successful when being implemented by the American Geophysical Union when they attempted to diversify their reviewer pool.

To embrace and encourage reviewer volunteers to diversify our reviewer pool

We often get requests from the research community to be considered as a potential reviewer for research articles that may be relevant to them. Currently we do allow people to volunteer to review for us, but this request is not as pronounced as it could be in our reviewer guidelines.

With this in mind, we will be creating a dedicated volunteer section that will enable us to very clearly encourage all researchers to put the names forward for consideration. This section will be explicit in that we would like to include researchers of all genders, researchers from any career stage and researchers from all ethnic backgrounds. By being truly inclusive and actively encouraging everyone to have an equal opportunity to be involved, we hope to increase and diversify our reviewer pool and allow our authors to make a more balanced choice when they come to select their reviewers.

Collecting information on gender and ethnicity from authors and reviewers

Throughout the week we have provided and made our analysis open on the gender and global distribution of our reviewers. However, we feel this data is not as robust as we would like. All of our gender data collection was collected using Gender API, which provides information on whether a first name belongs to someone who is male or female. This has limitations and uncertainty in the data due to a level of ambiguity in the data sources it uses to identify gender. Furthermore, our global distribution data is based on country affiliations, however this doesn’t effectively portray the ethnicity of an individual, all it does is provide information on where they work.

Therefore, what we feel is needed is the collection of real data provided to us directly from our authors and reviewers. Going forward we are going to give the opportunity for authors and reviewers to voluntarily provide us information about their gender and their ethnicity. By doing so, we will be able to collect more robust data so we can really understand where we need to improve as a publisher with regards to diversity and inclusion.

The power of a collective effort

Whilst we at F1000 have both a responsibility and vital role to play in ensuring that all voices of the research community are heard, real and lasting positive changes is undoubtedly dependent upon a systematic and collaborative effort between the research community at large. Once, we as an industry turn this corner shoulder-to-shoulder together, the standard of transparency, ethical standards of conduct and quality of publication will become readily transparent. Peer Review Week 2018 has certainly been an excellent platform and starting point for this, but we’ve still got a long way to go.