blog

A few weeks ago, a group of Georgetown, McDonough School of Business MBAs contacted F1000Research to ask about our ongoing campaign to encourage the publication of negative results. They’re trying to develop a way to make it easier for researchers to share negative data. Below, you can find out more about the project from Dennis Lee. (In related news: We’ve extended the deadline for free submissions of negative results papers until the end of September.)

The genesis of our project has its roots in my time as a graduate student in plant molecular genetics over 15 years ago, working on an experiment which I later found out only through personal communications could never work using the techniques I was employing at the time. Had that information been published somewhere, perhaps I could have saved myself months of work and frustration. Though I no longer am an active research scientist, many of my friends in the research world have also tossed around the concept of a journal of negative results.

Fast forward to mid-July. During a class on entrepreneurship in the Georgetown MBA program, my classmates and I decided to take another look at this idea and see whether there is a viable business buried somewhere in there. We went out and talked to researchers across a wide variety of fields, journal editors, and more. In particular, we wanted to verify whether this was an actual need, and to find out what might be the roadblocks to making it happen. In general, the consensus appears to be that yes, a journal or database of negative results would be useful, but finding the appropriate incentives to encourage researchers to commit their results is non-trivial.

A couple of the key observations that came to the forefront in our interviews:

1. The opportunity cost of not having publically available negative or null results is different in different fields. For example, in fields heavy on computational data analysis such as bioinformatics, space science, climatology, or accounting & finance, the cost to run another experiment is less than in fields such as molecular biology or geophysics where there may be substantial costs in both time and money for each additional experimental run to collect data. This has direct implications on our focus; it would appear to make the most sense starting in areas where costs are high in order to maximize the incentive to contribute.
2. It’s not just about negative and null results. There are other categories of research results that while important, don’t meet the unwritten bar of a “minimum publishable unit”. A corollary to that is that researchers don’t want to spend the same amount of time writing up negative/null/partial results that they would a manuscript for a traditional journal.

Unsurprisingly, we aren’t the first people to think of the idea of a journal of negative results – for example, see F1000Research’s own campaign to encourage publication of negative results, Rejecta Mathematica, or the Journal of Negative Results in Biomedicine. However, in the 15+ years I’ve been thinking about this idea, no one has really figured out the best way to make it work. Hopefully there’s room for one more entry!

We’re planning on taking this forward – we’ve gained enough validation of the idea to understand that at least in certain fields a searchable database of previously unpublished research would be of great value, assuming certain hurdles like verifiability, ease of use, and incentives to contribute could be overcome. The real trick will be to find the right model (we have some half-baked ideas), the right backers, and most importantly, the right research community. Building a critical mass of users is key to most ventures, and ours is no different. Users must be willing to contribute their work and be part of a collaborative community. Competition will always be a part of research, but collaboration has to be as well.

Follow our progress at https://journalofnegativeresults.tumblr.com; if we manage to launch a beta product, we’ll be sure to post the information there. We’re open to suggestions, and would love to talk to researchers and research-adjacent professionals of all stripes, from all fields, in both academia and industry to gather more information. So if you have any ideas, or know anyone we should be talking to, please point them in our direction.