Filling in the Scientific Record: The Importance of Negative and Null Results

PLOS strives to publish scientific research with transparency, openness, and integrity. Whether that means giving authors the choice to preregister their study, publish peer review comments, or diversifying publishing outputs; we’re here to support researchers as they work to uncover and communicate discoveries that advance scientific progress. Negative and null results are an important part of this process. 

This is something we agree on across our journal portfolio — the most recent updates from PLOS Biology being one example– and it’s something we care about especially on PLOS ONE. Our journal’s mission is to provide researchers with a quality, peer-reviewed and Open Access venue for all rigorously conducted research, regardless of novelty or impact. Our role in the publishing ecosystem is to provide a complete, transparent view of scientific literature to enable discovery. While negative and null results can often be overlooked — by authors and publishers alike — their publication is equally as important as positive outcomes and can help fill in critical gaps in the scientific record. 

We encourage researchers to share their negative and null results.

To provide checks and balances for emerging research 

Positive results are often viewed as more impactful. From authors, editors, and publishers alike, there is a tendency to favor the publication of positive results over negative ones and, yes, there is evidence to suggest that positive results are more frequently cited by other researchers. 

Negative results, however, are crucial to providing a system of checks and balances against similar positive findings. Studies have attempted to determine to what extent the lack of negative results in scientific literature has inflated the efficacy of certain treatments or allowed false positives to remain unchecked. 

The effect is particularly dramatic in meta-analyses which are typically undertaken with the assumption that the sample of retrieved studies is representative of all conducted studies:

“However, it is clear that a positive bias is introduced when studies with negative results remain unreported, thereby jeopardizing the validity of meta-analysis (25, 26). This is potentially harmful as the false positive outcome of meta-analysis misinforms researchers, doctors, policymakers and greater scientific community, specifically when the wrong conclusions are drawn on the benefit of the treatment.”

— Mlinarić, et al (2017). Dealing with publication bias: why you should really publish your negative results. Biochem Med (Zagreb) 27(3): 030201

As important as it is to report on studies that show a positive effect, it is equally vital to document instances where the same processes were not effective. We should be actively reporting, evaluating, and sharing negative and null results with the same emphasis we give to positive outcomes.

To reduce time and resources needed for researchers to continue investigation

Regardless of the outcomes, new research requires time and financial resources to complete. At the end of the process, something is learned — even if the answer is unexpected or less clear than you had hoped for. Nevertheless, these efforts can provide valuable insights to other research groups.

If you’re seeking the answer to a particular scientific question, chances are that another research group is looking for that answer as well: either as a main focus or to provide additional background for a different study. Independent verification of the results through replication studies are also an important piece of solidifying the foundation of future research. This also can only happen when researchers have a complete record of previous results to work from. 

By making more findings available, we can help increase efficiencies and advance scientific discovery faster. 

To fill in the scientific record and increase reproducibility

It’s difficult to draw reliable conclusions from a set of data that we know is incomplete. This lack of information affects the entire scientific ecosystem. Readers are often unaware that negative results for a particular study may even exist, and it may even be more difficult for researchers to replicate studies where pieces of the data have been left out of the published record.

Some researchers opt to obtain specific null and negative results from outside the published literature, from non peer-reviewed depositories, or by requesting data directly from the authors. The inclusions of this “grey literature” can improve accuracy, but the additional time and effort that goes into obtaining and verifying this information would be prohibitive for many to include.

This is where publishers can play a pivotal role in ensuring that authors not only feel welcome to submit and publish negative results, but to make sure those efforts are properly recognized and credited. Published, peer-reviewed results allow for a more complete analysis of all available data and increased trust in the scientific record.  

We know it’s difficult to get into the lab right now and many researchers are having to rethink the way that they work or focus on other projects. We encourage anyone with previously unpublished negative and null results to submit their work to PLOS ONE and help fill in the gaps of the scientific record, or consider doing so in the future.