faculty of 1000

Post-publication peer review

Archive for November 16th, 2009

On being systematic

Posted by rpg on 16 November, 2009

Over on another planet blog Darren Saunders asks what is an Associate Faculty Member (AFM).

There was some sales training on this subject last week and I sat in, so I should know. I’ve also been re-writing the FAQs for the new f1000 website and have just realized that there isn’t an FAQ relating to AFMs there, either. (Meta: how many times does a question have to be asked before it becomes ‘frequent’?) So, let’s have a stab at explaining it.

As you may or may not know, what Faculty of 1000 does is publish short reviews of the scientific (currently biology and medicine) literature. How this works is through our eponymous Faculty of over 5000 top scientists and medics, all over the world. These people are principle investigator level or higher. When they read a paper that they consider interesting, important, or otherwise worthy of wider recognition they write a review (or ‘evaluation’), assign a score (or rating) to the original article, and submit it to our editorial team (usually via a web interface). The piece is then edited in the usual way, coded to appropriate sections (i.e. sub-disciplines), and published on the website at f1000biology.com or f1000medicine.com, depending on the specialty of the contributing Faculty Member.

This system has been pretty successful for a few years now, and we know that people really like the service (because they tell us!). It lets scientists and medics see very quickly what’s happening in their own field, and rapidly get at what’s considered important in other communities (whether simply out of interest or because they’re moving into unfamiliar territory). Identifying important papers quickly and easily gauging the opinion of a field easily are not trivial tasks: f1000 is intended to help everyone, from students through to vice-chancellors, achieve this.

Critically, choice of articles to review is left entirely to the Faculty, and may come from any journal. Any journal: even the Harvard Business Review. Naturally there are a high proportion of articles from the usual suspects—Nature, Cell, NEJM, etc.—although about 80% come from ‘second tier’ or less popular journals (he says, desparately avoiding the ‘I’ word).You might expect this, seeing as certain journals review editorially before a paper goes anywhere near peer review, and actually are quite successful at it.

In a sense, we don’t care about the providence of the articles reviewed at f1000. If they’re good, we want to know (and ‘good’ means 1-2% of the current eligible literature). However, there are a lot of journals publishing good stuff, and how do we know we’re scanning the right ones if we’re just leaving it to serendipitous reading by the Faculty?

Enter the Associate Faculty. Currently about a thousand Faculty Members have one or more Associates: less senior members of their lab or practice (which can mean anything from a post-doc to a PI in their own right). Once a month we send these Associates a table of contents from two journals: one general, one ‘specific’; both self-nominated. The Associate checks the table against their own reading, and selects articles that they have already read that they will review. They also let us know if there are any articles that they think should be reviewed but that they will not do themselves: these then go into a ‘pot’ which we send (a couple of weeks later) to Associates who haven’t committed to producing a review that month.

When the Associate commits to reviewing an article, it’s pretty much between them and their Faculty Member as to how it’s handled. Sometimes the Associate will do the bulk of the writing, other times the Faculty Member will. In either case, the full Faculty Member has to approve the evaluation and has final say—they are the corresponding author.

We cover, at the last count, about 660 journals in this fashion. We’ve asked the Faculty to tell us what journals they think should be scanned in this scheme, and eventually we’ll be covering over a thousand different journals. This does not mean that we won’t be evaluating articles outwith this ‘core’ of journals: Faculty Members have complete freedom to evaluate papers regardless of where they are published. Our Associate Faculty help them identify the good stuff, and we help them to choose by providing the tables of content with a selection system (somewhat arcane, but we are working on it). The buzzphrase is ‘systematic and comprehensive’: we’re certainly systematic and are working on the comprehensive.

Hope that clears some things up for Darren.

Read the rest of this entry »

Posted in FAQs, FMs | Tagged: , , | Leave a Comment »

Doing science to it

Posted by stevepog on 16 November, 2009

I often run updates on the news stories we put out from f1000 that are picked up in the media. Most of the time it’s good coverage, occasionally (as the mainstream news media is wont to do) they misinterpret the research and then some poor reporting is cut-and-pasted on blogs around the world.

In the UK, the Daily Mail and Daily Express are routinely derided for their page 1 mix of medical scare stories and unfounded cancer wonder drug revelations. US readers will have their own examples of media outlets that think ‘doing science’* to a story (eg. quoting stats from a straw poll by a first-year researcher from a low-grade institution) makes it factually correct.

The subject of the discussion today was this release on an F1000 Medicine review of research into how first-time mothers who have long-term exposure to a father’s semen have a lower risk of preeclampsia and generally healthier babies. The title was catchy, the science was sound and the story related to sex so it was bound to gain traction and it did, from the UK Telegraph, Daily Mail, Sky News Australia and a raft of medical and science blogs.

A blogger on Blisstree says using the word ‘faithful’ in the headline was incorrect and if you look into the detail of the study, the main focus is on the duration of sexual relationship between the mother and biological father, while the number of sexual partners is a sidenote. So we take it on the chin that the word ‘faithful’ was misplaced but the Blisstree blogger was wrong to say that many news stories (and us, by association) got the facts wrong.

It pays to be careful when making big statements: sometimes the difficulty is crunching the original title (in this case, Duration of sexual relationship and its effect on preeclampsia and small for gestational age perinatal outcome) into one that is factually correct and easy to understand. But not so simple that the original message gets lost and people start believing that tea cures cancer or other such nonsense.

*When it comes to ‘doing science’ to anything, this is the only time it’s applicable (thanks to Dresden Codak):

Doing science to it

Image courtesy of: http://topatoco.com/

Posted in Communication, Journalism, f1000 | Tagged: | Leave a Comment »