Evaluations and Jeopardy
| 16 March, 2011 | Richard P. Grant |
3
|
|
While checking out Thomson Reuters’ Science Watch I checked to see if their ‘hottest’ scientists from last year (citation-based) were anything like the ones we might choose if we were to have an F1000 top scientist list (evaluation-based). Their number one slot for 2010 goes to Eric S. Lander, who contributed to ten ‘hot’ papers as measured by citations. There’s a fair number of papers to have your name on in one year, let alone to get any of them cited, but genomics and genetics is a pretty fast-moving field to be in.
Lander contributed to seven papers in 2010 that attracted F1000 evaluations. I guess our Faculty weren’t that impressed by the rest–but that’s still a pretty amazing achievement. Soon I hope to be able to bring you our own rankings of scientists and medics in each field, and then we’ll be able to make some sensible comparisons between citation-based and evaluation-based statistics.
Science Watch updates its ‘hot’ papers every two months. And naturally, citations take a little while to accrue: even in the rare instance of citing a newly published paper in your own manuscript while it’s in press there’ll be a reasonable delay before it can be counted (and it would be arguable, in these circumstances, whether the citation means anything). F1000 evaluations are inherently a lot faster: it’s much quicker to publish a short review of a paper than to incorporate its findings into a whole new manuscript. So this means that you should be able to detect trends in biology and medicine by keeping an eye on what’s coming out at F1000—something I’m planning how we might formalize.
Aside from all that, I checked to see if Lander had anything published and evaluated this year already. And yes, he has:
High-quality draft assemblies of mammalian genomes from massively parallel sequence data, pleasingly Open Access in PNAS1.
I’ve already said that F1000 is faster than citation-based metrics. But there’s another advantage, too: take a look at how Tony Long, at UC Irvine, starts his evaluation:
“I will take Nextgen sequencing for 200 Alex.”
Now, come on. What journal is going to let you have that much fun writing (or reading) a paper?
|
|
previous post
There is another side of these hot stories, though. Besides the citations as soon as the papers are out, or just a little while after they are out, there is also how long they keep being cited. In the ISI Journal of Citation Reports (JCR) this is measured by the Cited Half-Life (CHL) of the journals. If it is more than ten years, then what is published there is considered to be almost “eternal”. Usually, the highest the Impact Factor, the lower the CHL of a journal, which means that the decay time of this stuff is very rapid. Take Cell Biology in the JCR, and rank the journals with the CHL: just 10 have >10, and the highest IF in this group is 2.1, a ridiculous score in that field. Now take Zoology. 51 journals have more than 10 years of Cited Half-Life. The highest IF in this flock is 3.7. I have a provocative proposal: why not invent an integrated measure of journal value, obtained by multiplying the IF by the CHL?Strange enough, we evaluate scientific performances based solely on IF, i.e. on impacts (in terms of citations) within 2 to 5 years since papers are out, and then we forget about them. The rest seems worthless. But the JCR gives also other figures, why nobody cares? It is like comparing the performances of the athletes who run the 100 m with those of the guys who run the marathon, and then say that the marathon athletes are worthless. As for myself, I try to highlight the classics, every once in a while, in my evaluations for F of 1000. Of course there is no section for taxonomy in F of 1000 (the science of biodiversity exploration is dying, in the era of biodiversity) but the articles in that field are eternal, and Linneus 1758 is still a citation classic. Maybe it might be worth-while opening a section for this stuff. Old articles that the young guys should read, because they are still full of knowledge, and wisdom (the articles). I even have a name for such section: the cellar. Where you keep the good wine, and the older it gets, the better it is. This would also prevent the re-invention of the wheel every two to five years, as it is the case for jellyfish blooms, nowadays.
Nando, a thoughtful comment, as always.
I’ve been toying with the idea of a ‘Classics’ classification within F1000 for some while, although we haven’t gone anywhere with it yet. Your idea that it should be something for the neophytes to read is a good one, as is the potential name. Don’t know whether it should be an entire Faculty (but why not?) yet, but I’ll see if we can get somewhere with it.
I agree with Nando Boero.
Really current fashion is very important factor influencing on every citation rate (citation index)….but this factor is very changeable….
principal new idea/ fact is out current fashion all time…probably it will be an object of future fashion….