Benoit’s comment on Jenny’s blog reminds me of the time that I was scooped, but not because my boss was carefree with data. I’ll tell you about that some other time, though, because there’s an another issue he mentions that’s recently become very pertinent.

One of the issues that crops up occasionally at the day job is retractions. Because we highlight the ‘best’ in the biological and medical literature it’s critical that we do something whenever a paper is retracted, for whatever reason. These can be very good reasons, such as conclusions becoming untenable in the light of new evidence, or irreproducible results; or because of deliberate or accidental fraud or misconduct.

There’s nothing inherently wrong with retractions: it’s part of science’s self-correcting mechanism (although we could all use a little more openness). Either way, when a paper is retracted it doesn’t just disappear: the paper itself and the retraction text remain part of the scientific record. As, indeed, do the evaluations we publish. We (i.e. f1000) need to maintain our records for the sake of posterity; we don’t simply delete evaluations of retracted papers.

Now that’s out of the way, Benoit’s comment about PIs:

the PI’s job in those cases is to talk to the competition, figure out where everyone is at [...] and make arrangements for co-publication, for example. It is no accident that a recent issue of Nature had 5 papers on the same iPS cell result, or that the cover of Nature was shared years ago by two articles on the role of lunatic fringe in chick limb development

reminds me of a story that started—at least in public—three years ago and is only just nearing closure.

Not being particularly interested in immunology, my response to the publication in Nature of three papers (by Janssen, Wiesmann and Abdul Ajees et als) reporting the structural basis for the activation of complement C3b elicited little more from me than an “ooh, that’s pretty” together with a raised eyebrow at an 80 Å shift in conformation. (I should add, here, that it’s not surprising to see three such major pieces of work published simultaneously in the same journal. As Benoit says, PIs talk—with each other and with journal editors—and publication can be delayed a little, or hurried. Which I suspect has not a little bearing on the current story.)

I would have paid it no further attention were it not for our soon-to-be departmental head, sometime in the waning of 2007, highlighting a rather intriguing Communication in Nature. Two of the authors on one of those three C3b papers, together with a couple of big guns in protein crystallography (one of whom was responsible for the gazumping I mentioned earlier), put their collective hand in the air and said, “Hang on, there’s something fishy with this third paper”:

We have reanalysed the data deposited by et al. and have discovered features that are inconsistent with the known physical properties of macromolecular structures and their diffraction data. Our findings therefore call into question the crystal structure for C3b reported by Ajees et al.

Them’s fighting words. The Communication gets a little technical (this is protein crystallography, after all), but I’ll try to explain it.
These guys decided, as you might expect, to compare the three structures. A sensible move, especially seeing as the published structures are somewhat dissimilar. Indeed, the Ajees structure is missing a huge chunk of molecule!

The coordinates do not form a connected network of molecules in the crystal lattice. The crystal structure forms layers that are separated by a large void in the c-direction (a slab of about 30–40 Å thick that spans the entire unit cell).

Sometimes it’s actually difficult to ‘find’ molecules in large, complex protein structures—if the software you use doesn’t assign electron density correctly for some reason, or the phases aren’t complete, you can fail to join the dots properly, and in the rush to publication this can be missed. So these guys took the deposited data and tried to solve the structure again, looking for the putative missing protein molecule. And they didn’t find it.

Furthermore, they noticed other physically implausible features such as no data indicating that the crystals contained water. In other words, this protein was in a vacuum; which might make theoretical physicists happy but isn’t really consistent with anything we know about biology (and Henry, you can stop the Nature abhors a vacuum joke right now).

_Killer figure_

They also found that the R factor, essentially a measure of the ‘goodness’ of a protein structure, doesn’t behave you would expect if no water was present in the calculation. Another measure of structure goodness, the B factor, looked odd too. This is a measure of how much any given atom might be expected to move; you might predict that atoms on the outside of a protein have more freedom than those inside: so the B factor should vary along the sequence. Right?

Eh. Right. In the Ajees structure, the B factors are pretty much the same across the entire length of the protein, which when you consider that the structure has vast swathes exposed to solvent (or vacuum, perhaps) is somewhat puzzling.

_somewhat puzzling_

In brief, the structure is just too good:

We think that these physically implausible features undermine the validity of the model presented by Ajees et al. and the deposited diffraction data from which it derives. Only when the experimental diffraction images are made available can the deviating C3b model be either verified or falsified.

There’s a bit of a pathetic response from Ajees and co., but basically we (as in the crystallography) waited to see what would happen next.

<eats popcorn>

What did happen next was a veritable storm on the CCP4bb mailing list, kicked off by Eleanor Dodson and titled The importance of USING our validation tools. That’s all it took for Randy Read (one of the Big Four) to weigh in with

Originally I expected that the publication of our Brief Communication in Nature would stimulate a lot of discussion on the bulletin board, but clearly it hasn’t. One reason is probably that we couldn’t be as forthright as we wished to be. For its own good reasons, Nature did not allow us to use the word “fabricated”.

That, and the necessity of waiting for Ajees et al. to respond, is probably why it took eight months for the Communication to be published. Anyway, rather than clutter up this space, I commend to you this zip archive of the CCP4bb discussion, kindly collated by Hari Jayaram.

Why am I talking about this now? Because, as reported by Iddo Friedberg, the University of Alabama at Birmingham has (finally) requested that no fewer than eleven structures, represented in nine papers, from the lab that published the apparently dodgy structure be expunged from the Protein Data Bank:

After a thorough examination of the available data, which included a re-analysis of each structure alleged to have been fabricated, the committee found a preponderance of evidence that structures 1BEF, 1CMW, 1DF9/2QID, 1G40, 1G44, 1L6L, 2OU1, 1RID, 1Y8E, 2A01, and 2HR0 were more likely than not falsified and/or fabricated and recommended that they be removed from the public record.

My spy in Sydney tells me that the Journal of Biological Chemistry has retracted one paper, and that the Journal of Molecular Biology and Acta Crystallography. He also says that Birmingham (Alabama) were slow in informing all the parties, which has delayed final actions. I also understand that journals are moving towards requesting that raw diffraction images (even if they’re as bad as Stephen’s) rather than just structure factors be deposited somewhere, so that we can all keep a big, brotherly eye on each other. And finally, “it is interesting that [the head of the lab] still claims he is innocent”.

It’s funny, that two years ago I suspected that Nature received manuscripts from Janssen and Wiesmann in reasonably quick succession, and knowing that the Ajees lab was working on the same thing poked them with a “Have you got that structure yet?”-type call. Alternatively, I thought, perhaps the Ajees group had seen presentations by the other two and realized they were about to be pipped. In either case, I reasoned, they might have had a structure almost ready, or poor crystals, and maybe they made something up.

Not good, not ethical; but possibly, just possibly, understandable. But to apparently fabricate eleven structures? That’s seriously bad, and implies a level of collusion and—well, let’s be frank here—organized conspiracy unthought of outside certain labs in South Korea.

In the meantime, as soon as the news from Alabama hit my screen, I pulled up the f1000 website and searched for the implicated articles. I’m pleased to say that we now reflect the current state of the scientific record (as you
may verify for yourself at this free link).
Read the rest of this entry »