blog

A recent Opinion Article published in our Antibody Validation Gateway by Jan Voskuil (JV), Aeonian Biotech, discussed the challenges in validation of research antibodies. It was openly peer reviewed by Michael Weller (MW), Federal Institute for Materials Research and Testing (BAM), Germany. In this blog, both discuss these challenges, the reproducibility crisis and open data.

How does the poor conduct of commercial antibodies contribute to what is known as the reproducibility crisis?

 

JV: The poor conduct of antibodies can contribute to the reproducibility crisis in different ways. The scientist’s choice of antibody, to optimize and develop a required immunoassay, will be known as good or bad only after a significant period of invested time and experimentation. Consistency and robustness will eventually decide if the right antibody was picked. A weak antibody might work well in the beginning but its performance may deteriorate over time. A good working antibody in a validated assay cannot be easily replaced by any other antibody generated to the same protein. In fact, when the batch of the validated antibody changes, the new batch needs revalidation in the same assay. This aspect is often and easily overlooked. Also, the integrity of the antibody can be impaired by poor storage or by adverse tr

ansportation conditions, thus rendering the product unfit without having to dismiss the

catalogue number or the batch on trial. Thus, even when the catalogue number and clone number has been properly reported, the above-mentioned factors may jeopardize efforts to reproduce published results. And of course, also a good antibody can fail under poor scientific conduct.

MW: Yes, I agree. The reproducibility crisis is complex and not only limited on poor antibodies. However, using undefined, uncharacterized and unvalidated antibodies in a research project makes all subsequent results invalid. Therefore, high-quality antibodies are a prerequisite of good research in all fields using antibodies as a tool.

Antibody validation involves many stakeholders – manufacturers, resellers, researchers, publishers etc. What do you think each party can do to help improve antibody validation? And how do you think they should be working together?

 

JV: As pointed out in my article, validation relies heavily on the context in which the antibody is being used by the researcher. It is essential to validate the antibody in the application it is required for and to have the antibody compared between expressing and non/low expressing cell types. Because the required cell type will be different from project to project, such validation cannot be asked from the vendor or manufacturer. The supplier will have to demonstrate that the antibody is fit for the claimed application by showing convincing data on their product sheet. However, validation on epithelial cell line HeLa, comparing wildtype expression levels with knocked-down levels through siRNA, is great in theory but may not be convincing enough data in practice, for example when the antibody is required in research on the central nervous system. In addition, publishers need to demand catalogue specifics for each antibody used in the described experiments.

“As long as they [funders] tolerate sloppy proposals, research reports and publications, some researchers will take the fast lane and ignore even basic quality criteria.” Michael Weller

MW: This is true, indeed. I also want to motivate funders who have a very strong position in science. As long as they tolerate sloppy proposals, research reports and publications, some researchers will take the fast lane and ignore even basic quality criteria. There needs to be more strict implementation and enforcement of ethical guidelines – although this is standard in many funding organizations – and basic quality rules.

Some antibody manufacturers provide extensive and open data. How important do you think data sharing is in antibody validation research?

 

MW: The idea of Open Data is crucial for any progress in this field. Any activities of manufacturers and resellers in this direction are highly laudable.

“I completely agree that full and non-negotiable disclosure is the only option in science.” Michael Weller

JV: There is a dilemma among many manufacturers in showing everything (for example the sequence of the immunizing peptides, or the CDRs of recombinant antibodies) as there is always the risk of stealing/copying Proprietary Information (PI)  to be used for the trade of competing products by others. I am passionately in favour of an open data culture, the threat of such PI-theft will prevent this culture from from taking hold until global regulation prevents this practice. I believe compulsory disclosure is the only way to rule out such threats.

MW: De-novo sequencing of an antibody based on mass spectrometry is a nearly a routine task these days. Therefore, non-disclosure will only harm the clients; anyone wants to steal/copy PI  can easily overcome this “protection through obscurity”. I completely agree that full and non-negotiable disclosure is the only option in science.

F1000Research operates an open peer review model and all correspondence between authors and reviewers are published alongside articles. How did you find having peer review conducted in this way, both as reviewer and author?

 

“A very important factor in the reproducibility crisis is the failed peer-review system. It fails due to lack of transparency.” Jan Voskuil

MW: At first, I felt a little bit uncomfortable to see my referee report published. This may be only a question of getting accustomed to a new and unfamiliar format. A more interactive and open communication between author and referee should lead to a more constructive discussion and finally to a significant improvement of the quality of any paper. This format also has some advantages for the referee to make his/her efforts more visible. Now the referee becomes more involved in the process of the making of a paper and is not a simple opponent without much responsibility.

JV: A very important factor in the reproducibility crisis is the failed peer-review system. It fails due to lack of transparency. The transparent peer review model, used by F1000research means that referees will have to come up with valid reasons to dismiss a paper and they cannot use false arguments while being hidden behind anonymity as can potentially happen in a closed peer-review system. The risk of accepting poor papers, because the referees and authors support each other no matter what, is also reduced by the transparency. Having the referees openly included in the publication process improves the quality of the article and thus increases the reproducibility in science.

In his review, Michael says that he doesn’t think that including catalogue numbers in published articles is enough and that a clone number or real antibody ID should be also included. How easy would this be to implement? Who is responsible for driving this change in practice?

 

JV: It is not so easy as it seems. The difference between a catalogue number and a Research Resource Identifier (RRID), as offered by antibodyregistry is hard to see and the two appear synonymous. In contrast to good intentions, the RRID does not group identical antibodies offered by different brands and so one can find the same antibody under different RRID entries, depending on which brand it carries. This is true for both polyclonal antibodies (pabs) and monoclonal antibodies (mabs). The same clone number is carried by different catalogues, and each brand is listed separately. Some traders may generate a sub-clone and rename the clone, thus making its origin obscure. I would prefer to see, next to the catalogue number and clone number, the batch coding for both pabs and mabs specified both on product sheets and in publications. Once publishers start to refuse papers when such information is not specified, the pressure to improve will increase.

“Once publishers start to refuse papers when such information is not specified, the pressure to improve will increase.” Jan Voskuil

MW: Jan’s objections against the RRID offered by Antibodyregistry are valid. In this database, the same antibody clone can be assigned to different IDs, which is exactly the opposite of what an antibody ID needs to be. As an example, the clone AD4G2 was assigned to the RRIDs AB_10540451 and AB_2051866, both from the same supplier! I do not agree that a catalogue number and a (true) antibody ID should be synonymous. The RRIDs currently offered by Antibodyregistry are only hidden catalogue numbers and hence of negligible additional value. A catalogue number can change at any time and if a company is taken over by a larger supplier or goes out of business, the old catalog numbers might disappear without any notice and cannot be regained. A true antibody ID needs to be unique, will never change and will be invariantly connected to a single product of a hybridoma or other clone, ideally with a defined sequence. I completely agree that batch numbers of the antibodies are needed in any case to identify problems in the production or storage of antibodies.

Jan Voskuil provides consultation through Aeonian Biotech to relevant industries and their customers regarding revised quality requirements for research antibodies in the aftermath of recent international meetings on this subject. He also provides antibody testing and validation services through AVC Biotech. Jan has been the driving force behind the growth and quality of the Everest Biotech antibody catalogue, transforming it to a profit-generating enterprise with increasing frequency and amounts of dividends.

Michael G. Weller is the Head of Division 1.5 Protein Analysis of the Federal Institute for Materials Research and Testing (BAM), Berlin, Germany. He was a postdoctoral fellow at Ciba-Geigy, Basle, Switzerland, developing online biosensors based on fiber optical principles. His research group at the Technical University of Munich explored bioanalytical methods, such as protein microarrays and assays for the detection of cyanobacterial toxins. Today, his research interests are focused on the field of antibody development, immunoassays, affinity chromatography, quantitative protein analysis, peptide and amino acid analysis and biosensors for environmental, food, diagnostic and safety applications, such as the sensitive detection of explosives.