This rapidity of evaluation is part and parcel of the value of F1000 to researchers. Obviously we can’t evaluate as soon as a paper is published, but we’re still pretty fast–fast enough to make F1000 a crucial part of your literature search strategy. And if you’ve already read a paper that has just been evaluated, then the chances are it’s still fresh in your mind and the evaluation text itself will add to, or maybe modify, your thoughts.

On the other hand, F1000 evaluations also go back a long way. Certain F1000 Members seem to delight in finding and bringing to our attention historical nuggets. Our first evaluation (at least, according to the user-facing browse page…) is this one on morpholino siRNA inhibition of zebrafish genes, back in 2000. “It really works!” evaluating Member Michael Brand said of this now-commonplace tool. But some Members have really looked peered into the mists of time, pulling out papers from the 1950s, the 1940s, and even from as far back as 1918. This last is from our old friend Björn Brembs, and has the remarkably modern-sounding titled The reactions to light and to gravity in Drosophila and its mutants, authored by Robert Stanly McEwen of Columbia University (the evaluation is free. Help yourself).

But the oldest evaluated paper by far is from 1866, the impressively titled Über die Erweiterung von Arterien in Folge einer. Which brings us back to where we came in today, as I wrote about it in August 2009, in the context of generating speediness statistics for Faculty of 1000.

And on that note, today is my last day in the office at Faculty of 1000. I’m moving on to pastures new, although I will probably still be contributing the odd post here. I’ll still be in London, and you can follow my continuing adventures at my personal blog and on twitter. It’s been a blast; thank you for sharing the journey with me.

Buried deep within our About pages is a little badge you can put on your own website, or your CV, or whatever:

If you’ve had a paper evaluated at F1000 please feel free to use this badge and show off a little. If you’re really nice to me, I might even be persuaded to create a free link for your evaluation page, too.

We’re thinking about how we might use G+ in a way that’s beneficial to our Faculty Members as well as our users. Any thoughts, let us know. In the meantime, I’m rpgrant on google and you should be able to add me if you have an account–let me know who you are and I’ll add you to our F1000 Member circle.

PS I have some invites, too, if you’re interested.

Update: F1000 is on G+ at Naturally Selected.

In this analysis the authors “examine whether the magnitude of the effect sizes of biomarkers proposed in highly cited studies is accurate or overestimated”; and their conclusion? “Highly cited biomarker studies often report larger effect estimates for postulated associations than are reported in subsequent meta-analyses evaluating the same associations.”

Biomarkers are often used in studies as surrogates for ‘hard’ clinical endpoints such as death (the ultimate endpoint), heart attack, etc. The rationale being that showing an effect of an intervention on the biomarker will have a corresponding effect on the associated clinical endpoint.

Not only that, many biomarkers have been hastily incorporated into clinical practice, adding to the expense of diagnosis and, in some cases, intervention (think of being prescribed a drug because an elevation in biomarker X has been linked to disease Y). I don’t think anyone would argue that the use of properly validated and extensively researched biomarkers isn’t an extraordinary advance in personalised medicine. Such tools to enable earlier diagnosis will benefit all, but practitioners and patients alike should demand proper validation.

This paper raises another issue: surely it would be better to encourage a system where all the data from a study are released for analysis rather than just the main finding in a journal. Currently results may be enshrined as doctrine because of the logistical barriers to others examining or attempting to reproduce the data.

!F1000 plug alert!

Dr Ioannidis also says in the interview on the news@JAMA site that “Clinicians should keep in touch with the literature to the extent they have time to do so”. And how better to do so than by visiting F1000 regularly. There, I’ve said it!

PS. Since writing this I have read Dr Ioannidis’s article in this month’s issue of Scientific American where he calls for the full disclosure of scientific data… and then some!

Hidden away in our about pages there’s an explanation of how we calculate the F1000 Factor for evaluated articles. As it says there, Faculty Members rate each article ‘Recommended’, ‘Must Read’ or ‘Exceptional’. These correspond to a number, 6, 8 or 10, respectively.

For an article that attracts just one evaluation, the rating becomes the Factor. Things are a little more interesting for the quarter to a third of papers that get multiple evaluations. We wanted to boost the visibility of these articles (reasoning that there must be something extra special about an article independently selected by different Faculty Members) but we also wanted to maintain a scoring system that didn’t weight multiple evaluations unreasonably. So the old system, based on some bizarre averaging juju, wasn’t going to work—and we also didn’t think that the Factor for an article should ever go down as it attracts evaluations, which was possible under the old system.

We also wanted the new Factors to be easy to understand!

So what we do, for an article with multiple evaluations, is take the highest rating, and increment it according to the other ratings. For ‘Recommended’, ‘Must Read’ and ‘Exceptional’ the increments are 1, 2 and 3, respectively. Allow me to illustrate by example (drawn from recent articles on the site):

Quiescent fibroblasts exhibit high metabolic activity.
Three evaluations: one ‘Must Read’ (rating 8), two ‘Recommended’s (rating 6 each). Highest is 8, plus two increments of 1: Factor = 10.

Long noncoding RNAs with enhancer-like function in human cells.
Five evaluations: three ‘Must Read’s (8), two ‘Excellent’s (rating 10 each). Highest is 10, plus an increment of 3 for the second ‘Excellent’, and three * 2 for the rest. 10 + 3 + (3 * 2) = 19.

Highly efficient reprogramming to pluripotency and directed differentiation of human cells with synthetic modified mRNA..
Four evaluations, all ‘Excellent’. One 10, plus three times 3: Factor = 19.

You can verify others for yourself, as we now publish the individual Faculty Members’ ratings alongside their reviews. I hope you can now see how the Factor is based on what scientists and clinicians within each field think is interesting or important. It’s not escaped our notice that there might be power in the aggregation of these Factors, but I’ll talk about that next year.

This includes The Scientist magazine, and of course Naturally Selected, as you can probably tell.

There are a few wrinkles to sort out—our polls for example are MIA so I’ll have to go and find them somehow. Some navigation needs a tweak, but apart from that I think we’re in pretty good shape.

Please let us have your feedback (email me or leave a comment here) on the new design at The Scientist and F1000.com, and what you think to the new functionality.

This’ll happen just as soon as a last few tweaks are made: URL redirects, happy linkage, etc. Nip over to Naturally Selected anyway, as all content is duplicated there and has been for a while. Callum and I promise to keep you entertained as usual, and we’ll also be featuring guest and The Scientist writers.

Callum came to us from Cases Network last year and has been doing a lot of valuable donkey-work on our journal and institutions databases. He’s also been figuring out how to get our stuff into PubMed Central, which is why he’s a firm believer in the maxim XML is like violence: if it’s not solving all your problems, you’re not using enough of it. He will be writing about evaluations that catch his eye, and other newsworthy bits and pieces.

This is Callum’s first time blogging, so please make him feel welcome.

M’learned colleague Steve P is in North Carolina today, hobnobbing with the geeks at Science Online 2010. I decided not to go because I’d done quite a bit of travelling just before Christmas (admittedly not as much as last year), and having had a bit of a nightmare November personally, frankly I thought could do without the hassle.

However, those of you who are disappointed at not seeing me (hah!) will get their chance at the London equivalent of Science Online, Solo10. Yup, following the success of Science Blogging 2008 and Solo09 (not to mention Fringe Frivolous), we’re doing it again this year! I know this because I met with the inestimable Lou Woodley earlier today, along with Matt Brown and the Mendeley guys (and Martin Fenner by Skype), to discuss dates and all sorts of necessary weevils.

The programme is of course a mere glimmer in the distant sky, but I can tell you we’re looking at a two day (Friday/Saturday) event, and there will be a large collaborative component (indeed, we reckon that we can devote the Saturday morning to an ad hoc unconference). So there’s plenty of scope for plenary and parallel sessions, and you should start thinking about what you’d like to do/see. Matt Brown is likely to be running some pre-conference pub-crawls events, and we’re hoping to have a fringe pre-conference again (although I’m making no promises about me and Flip cameras, one way or the other).

Keep an eye on the Science Online London website,  and I’ll let you know about hashtags and whatnot in due course.

While in Crinan Street I was able to meet with Ian Mulvany (the brains behind Connotea) and discuss a couple more projects. First, he showed me what’s in store for the users of Nature Network: we’re getting MT4! This is a long-anticipated improvement in the platform there, and has acquired something of a mythical status. But I saw it!—on the staging server, at least.

The second thing is a little more ephemeral. I’m not at liberty to say much about it, but wouldn’t it be cool if you read an article in your favourite journal, saw that it had been evaluated on f1000, and could make a comment? And that comment then appeared next to the evaluation on f1000? Or maybe you could read an evaluation on f1000 and see what people were saying about that paper all over the web, and join in the conversation?

Like Google Sidewiki, but done properly?

Yeah.

Other things that have happened this week include us sending test data to PubMed Central. You can draw your own conclusions from that little snippet of information.  I’ve also spent quite a bit of time writing and polishing a press release about Sarah Greene. More on that next week; it’s now time to take our Dev team to the pub, methinks.

Or ‘Development’, as we call it in-house.

Hidden deep within the SNG intranet are a couple of wikis, which we use to keep each other informed, and collaborate on documents, etc. I discovered the Development wiki a couple of weeks ago, and while most of it is full of fascinating tips on how to use Velocity Macros to stop dirty little hobbits from throwing spanners into the fires of Barad-dûr, there’s the occasional gem such as

XML is like violence: if it’s not solving all your problems, you’re not using enough of it.

I just thought I’d share that with you.

* Sorry, I can’t say ‘Christmas’. It should be ‘Seasonal Carols’.

My  PR history is relatively short, just under three years, owing to finding it difficult to leave my previous  career in journalism (eight years across general news and sports writing in Victoria and the Northern Territory). I occasionally miss the buzz of putting newspapers together but also love the medical research/publishing PR caper as well.

@biggerpills tweeted that PR people should shout, not whisper, which I’m in agreement with when the story warrants. PR often gets a bad rap from those who relate it solely to political spin doctoring or the stereotypical movie/music celebrity publicity reps. Like good journalists, good PROs know the stories that need to be told and do their best to get the message out there. And when it relates to news on medical advances that could save lives, the story seems well worth telling.

Feel free to get in touch if you want more news on f1000, I’m at steve dot pogonowski at f1000 dot com. I’ll be filtering any messages with keywords such as ‘Ashes victory’ straight to junk mailbox (sorry, still in denial).

—SteveP

This is Stephen Pogonowski, who joins us from (most recently) the Medical Research Council, where he was Senior Press Officer. He’ll be handling our (whisper it softly) PR for us. And hopefully ramping up the content here and on Twitter. And working with me on Facebook and Linkedin in, too. Shh: he doesn’t know that yet, of course.

Steve is no stranger to blogging, and I’m hoping he’ll tell you all about himself. I’m now off to set up his account…

We’re the Faculty of 1000. We specialize in post-publication peer review. What this means is that we publish brief summaries of what our Faculty think are interesting, exciting, or otherwise noteworthy published articles in the biomedical literature. That’s the ‘post-publication’ and ‘review’ bits.

The Faculty consists of about 5,000 senior scientists and clinicians around the world. They are respected and authoritative within their specialties and disciplines. That’s the ‘peer’ bit. They are assisted by ‘Associate’ Faculty; less senior people (say, experienced post-docs) within the Faculty Member’s group. Associate Faculty are crucial to our scanning project, which I should talk about in a future blog entry.

I am F1000′s ‘Information Architect’. Essentially, I run the web-side service of F1000. Until March 2009 I was an active research scientist, and you can find some (out-dated, whoops) information about me on my personal website, and follow some random bloggy goodness at Nature Network.

Feel free to email me—richard.grant at f1000.com—or leave a comment here, if you have any feedback. I promise to read it, even if I can’t respond immediately. You can also find us on Twitter (@f1000).

Currently I’m the only one writing here and on Twitter, but I’m hoping to get more of the team on board soon.

Housekeeping

Look, we all know it’s a jungle out here. I’d love to read your comments here, and see you following us on Twitter. But we need to keep the spammers at bay, so when you comment, if it’s your first time you’ll go into the approval queue. Subsequently, if you are legit, your comments should appear straightaway. Sorry about the inconvenience.

I recommend you read our Policy document too, especially with regard to commenting. Nothing too unusual in there, but it keeps the lawyers happy. It’s likely to develop a little as time goes on. If you’re unsure about anything, please ask here.

See you around…