blog

Jason McDermott was one of the first people to publish online on F1000Research preliminary research methodology and data to gain peer reviews in the hope it would strengthen his R01 grant application (read about it here). Here he writes of his experience and how it went down with the grant reviewers.

McDermott is a senior research scientist at Pacific Northwest National Laboratory. He is also an F1000 Faculty Member and an author of two articles on F1000Research. You can find him on Twitter as @BioDataGanache and more of his comics at RedPenBlackPen.

---

 

The face of academic publishing is rapidly changing, with faster and more flexible methods for peer review publication becoming available. Likewise the format of the standard NIH grant application for most R01s and related mechanisms has shrunk from 25 pages to 12 pages for the research proposal. This presents a conflict for many researchers in that they would like to present their most recent and most relevant results as preliminary results for a grant, but there is very limited room to convey a story that has not been published already. One possible solution is to use a non-traditional publishing mechanism to accelerate the publishing of preliminary data for a grant. The F1000Research approach, which provides for post-publication peer review, is particularly suited to this approach. Submission of a manuscript to F1000Research quickly returns a digital object identifier (DOI) and a URL that links to the published paper to include in a citation in the grant application.

Earlier this year I applied this strategy to include the most relevant preliminary data for an R01 I was resubmitting for review (here’s my first post on this and you can read an overview of the science here). The idea was to submit the paper to F1000Research in time to be able to reference it in the grant. The F1000Research staff was very supportive of this idea and even wrote a blog post and press release promoting my idea. I was able to get the first step accomplished, submitting the paper and including a citation to it in my grant, with nearly no hitch (read about that here) – though I didn’t leave myself much time and so ended up sweating a bit.

The next step was to get it reviewed while I was waiting for my NIH review to happen (scheduled for early June of this year). This worked really well. I solicited about five reviewers and got two who were very quick about getting me reviews. The first paper reviews were very thorough and extremely helpful (you can read those here and here). It took me some time to get the requested changes implemented to the paper and resubmit the revised version (the final published version of the paper is available here). Unfortunately, this is where I think the process hit a major bump. Because I didn’t get my revised version submitted in time, the grant reviewers (very probably) only saw the first submitted version – not the vastly improved final version (see below). This was no fault of the reviewers or the publishing platform – no, the guilt lies solidly on my shoulders for not being faster about getting my revisions done.

Finally, the grant reviewers provided some feedback on my strategy and I give a more detailed overview of their inferred opinions in this post. In summary, the first reviewer was the most negative about this, actually writing (apparently with a straight face): “It is not appropriate to point reviewers to a paper in order to save space in the proposal”. They seemed to be somewhat annoyed, and seemed to imply that I was trying to get away with something by doing this. However, if the final accepted version of the would have been available to the reviewer I feel they would have been much softer on this point. The second reviewer had this to say: “The public reviewers’ comments on the paper itself raise several concerns with the method proposed and whether it actually works sufficiently well.” This certainly seems to be a concern that academics have with this model of publication –it is a bit scary to have reviewers’ critiques exposed for your paper. I really believe that this comment can be attributed to the grant reviewer not seeing the final version of the paper, where I’d addressed all those issues. The final reviewer had little to say about the process one way or another.

So, overall, the opinion of the grant reviewers was negative about this approach to preliminary data, but I really believe that this is mostly due to the final version not being available to them. My experience with the F1000 Research editors and staff, and the reviewers on the paper, was really great through this. I believe that this approach will become more common and hopefully someone can have better success than I did with it. My advice is to make sure you give yourself enough time (weeks not days) to account for unforeseen minor delays, with the initial submission as well as the revisions.