Working with protocols.io to improve reproducibility
12 March, 2018 | Lenny Teytelman |
|
|
Lenny Teytelman explains how protocols.io supports reproducibility and gives us some back story

Today, F1000Research is adding protocols.io to its author guidelines to encourage scientists to detail the recipes in the methods sections of their papers. The goal is to make it easier for readers to reproduce and build upon the published work.
Authors submitting manuscripts to F1000Research are encouraged to enter protocol details into protocols.io, issue a DOI, and refer to it in the paper’s Materials and Methods section. Once the article is published, the protocol will automatically link back to the manuscript with the full article citation. At any point after the article’s publication, the authors can create new versions of their protocols to share optimizations and corrections.
This integration feels particularly meaningful in light of our conversation with the founder of F1000 Vitek Tracz and the managing director Rebecca Lawrence in 2014. The day of our conversation, protocols.io had a grand total of 12 public protocols.
When we launched protocols.io four years ago, we had a clear mission: to give scientists an easy way to share and discover detailed research protocols, with the ability to keep them up-to-date with corrections and optimizations. We understood fairly well what functionality protocols.io would need to be useful to researchers. We realized that the design and user experience were critical. However, knowing what to build and how to build it is not enough; it is equally important to reach the scientists, to make them aware of the resource and encourage them to share their knowledge. Alas, as first-time founders, we were clueless about that outreach bit.
After three months of growing at a rate of 1-2 public/3-4 private protocols per month, I reached out in desperation to Michael Eisen, my PhD co-advisor and the co-founder of the Public Library of Science, asking for advice. I was hoping for some magic trick that would suddenly lead to a viral explosion. Michael had no such trick, but he told me that I must speak to Vitek Tracz: a visionary with a track record of innovation in science publishing, starting BioMed Central, F1000, and many other successful efforts.
As I hoped, Vitek responded to my request and validated the importance of our quest:
“Dear Lenny … I know a little of your project, and I have little doubt that there are many important issues to address with the availability of protocols, and to create a repository would be a good thing, though clearly not an easy task.”
Vitek Tracz, email correspondence, March 2014
He spent a significant amount of time on a Skype call with me and my co-founder/CTO of protocols.io, Alexei Stoliartchouk. Vitek and Rebecca were supportive of our effort, but they warned us that no single feature, no special button would suddenly make protocols.io a household name among researchers. They both stressed to us that to succeed, we would need to somehow obtain high-quality content: protocols that scientists need and trust.
We quickly learned just how right both Vitek and Rebecca were. We naively expected scientists to flock to the platform and start sharing their knowledge just because the resource exists. It became apparent that new scientists visiting protocols.io for the first time weren’t clicking on the “create new protocol” button; instead, they went to “search” and queried for their favorite methods. Seeing no results, they would just leave. The scientist were looking for content.
What followed our conversation with Vitek and Rebecca was a long period of startup hell. It was a desperate fight to survive, gain momentum, and somehow obtain that elusive content. We learned that partnerships with journals, funders, and reagent vendors were the path to high quality methods and, in turn, increased adoption.
Today, we have over 2,000 public protocols and are growing at a much healthier rate of 100 public/400-500 private protocols each month. Yet, we are still in an early stage where the vast majority of scientists are just discovering that protocols.io exists. This is why we treasure every new publisher that understands the importance of reproducible methods and connects the submitting authors to protocols.io. We are grateful to Vitek and Rebecca for their early support and advice and are excited to partner with them in improving the speed and quality of science publishing.
|
User comments must be in English, comprehensible and relevant to the post under discussion. We reserve the right to remove any comments that we consider to be inappropriate, offensive or otherwise in breach of the User Comment Terms and Conditions. Commenters must not use a comment for personal attacks.
Click here to post comment and indicate that you accept the Commenting Terms and Conditions.