Using expert recommendations to evaluate research outputs
26 January, 2017 | Liz Allen, Director of Strategic Initiatives |
|
|

Over the past decade, there has been a massive drive to improve the efficiency, and understand the impact, of research and Researchfish® was born from this. Developed by the UK’s Medical Research Council, and quickly adopted by all the UK Research Councils and a number of other major funding agencies and institutions across the world, Researchfish today provides the world’s most commonly used grant output and impact tracking platform. F1000Prime has now partnered with Researchfish to help track the impact of research outputs.
Building a portfolio of research outputs
Researchfish enables funding agencies and institutions to follow the development of, and build a portfolio of, the research outputs and outcomes linked to their support. At its most basic, the platform is a repository of funding-related outputs; its true potential, however, is in supporting decision-making around funding investments and how resources might be best employed to maximum effect. Bringing indicators of the value, use, re-use and potential importance of the reported research outputs together in one place can be hugely valuable to research funding agencies. And that is why the partnership between F1000Prime and Researchfish is so exciting.
Where F1000Prime fits in
The basic premise behind F1000Prime is that one of the important ways that researchers decide which research they should see as significant and potentially build upon, is through the advice of peers and experts. F1000Prime brings together the article recommendations of a virtual, independent Faculty of over 8000 leading life science experts from across the world on those research outputs that they think are particularly interesting, important and/or novel.
On average, 800 new articles are recommended each month, which corresponds to about 1.5% of all published articles in the biological and medical sciences. F1000Prime’s purpose is to be a guide to ‘what’s best in science’ – what do Faculty experts think that the research community should consult in their related field. Articles not recommended are not necessarily of lesser value of course, but a recommendation on an article provides an added ‘stamp’ of potential significance. An F1000Prime article recommendation is already an established indicator of potential research importance in its own right, and can complement quantitative citation-based indicators.
Recommendations feeding into evaluations
Currently funding agencies, research institutions and research evaluators have to tap into a range of sources to try to bring together a picture of research outputs alongside any relevant indicators of value and use; this process could be greatly simplified by linking relevant data together and build interoperability across various platforms. As a step towards this, from next month, Researchfish users will be able to freely identify any research outputs in their system that have been recommended by F1000’s expert Faculty, providing users with greater insight into the value and potential impact of much of the research that they have supported.
What next?
We at F1000 are keen advocates of the Leiden Manifesto’s approach to research-related metrics that highlights that it is important to use the ‘right’ research-related metrics in the right context. We also want to be part of the shift away from an unhealthy and erroneous reliance on journal-based metrics as proxy indicators of research quality, a reliance that has largely come about because of an absence of other robust and systematically available science-related indicators. We want all researchers to be encouraged to share and be rewarded for the true range of their research outputs – from articles and books, to data and datasets, through software to artworks – and be assessed by indicators that are tailored and relevant to their aims and that output. We hope that by adding expert recommendations into the mix on platforms like Researchfish, we can start to provide a new perspective on the potential ‘value’ of research and expand the reach, use (and re-use) and visibility of all the excellent work provided by our Faculty.
|