blog

A comment thread on one of The Scientist articles gives me the opportunity to mention F1000 Factors.

Hidden away in our about pages there’s an explanation of how we calculate the F1000 Factor for evaluated articles. As it says there, Faculty Members rate each article ‘Recommended’, ‘Must Read’ or ‘Exceptional’. These correspond to a number, 6, 8 or 10, respectively.

For an article that attracts just one evaluation, the rating becomes the Factor. Things are a little more interesting for the quarter to a third of papers that get multiple evaluations. We wanted to boost the visibility of these articles (reasoning that there must be something extra special about an article independently selected by different Faculty Members) but we also wanted to maintain a scoring system that didn’t weight multiple evaluations unreasonably. So the old system, based on some bizarre averaging juju, wasn’t going to work—and we also didn’t think that the Factor for an article should ever go down as it attracts evaluations, which was possible under the old system.

We also wanted the new Factors to be easy to understand!

So what we do, for an article with multiple evaluations, is take the highest rating, and increment it according to the other ratings. For ‘Recommended’, ‘Must Read’ and ‘Exceptional’ the increments are 1, 2 and 3, respectively. Allow me to illustrate by example (drawn from recent articles on the site):

Quiescent fibroblasts exhibit high metabolic activity.
Three evaluations: one ‘Must Read’ (rating 8), two ‘Recommended’s (rating 6 each). Highest is 8, plus two increments of 1: Factor = 10.

Long noncoding RNAs with enhancer-like function in human cells.
Five evaluations: three ‘Must Read’s (8), two ‘Excellent’s (rating 10 each). Highest is 10, plus an increment of 3 for the second ‘Excellent’, and three * 2 for the rest. 10 + 3 + (3 * 2) = 19.

Highly efficient reprogramming to pluripotency and directed differentiation of human cells with synthetic modified mRNA..
Four evaluations, all ‘Excellent’. One 10, plus three times 3: Factor = 19.

You can verify others for yourself, as we now publish the individual Faculty Members’ ratings alongside their reviews. I hope you can now see how the Factor is based on what scientists and clinicians within each field think is interesting or important. It’s not escaped our notice that there might be power in the aggregation of these Factors, but I’ll talk about that next year.