Negative Results: A Crucial Piece of the Scientific Puzzle
Scientific advance relies on transparency, rigour and reproducibility. At PLOS ONE we have always supported the publication of rigorous research, in all its forms, positive or negative, as showcased in our earlier Missing Pieces Collection. In this 10th Anniversary Collection, A Decade of Missing Pieces Senior Editor Alejandra Clark revisits this important theme and highlights a decade of null and negative results, replication studies and studies refuting previously published work.
Despite the value that these studies add to the scientific literature, they remain rare. Why? Some may question the value of publishing negative results or replication studies arguing that doing so in fact “litters” or “dilutes” the literature. We, however, believe that with the advent of ever more powerful search engines and automated data mining, this is an obsolete argument. Provided that the research question posed is meaningful, the study design appropriate, and the study rigorously conducted and reproducible, why shouldn’t those results be publicly available? In today’s era of social media and quick dissemination of information (both verified and unverified) the responsibility on scientists to report valid research irrespective of the results is as relevant as ever. Importantly, sharing results via a scientific publication prevents others from following the same erroneous paths or wasting valuable resources on duplicating efforts.
As to why so few studies reporting negative results are published, possible factors could include time restraints and the limited number of venues willing to consider these for publication. The “publish high impact papers or perish” culture means that some labs may find it inefficient to publish negative or null results given the drive to focus on positive findings. In addition, some scientists may be reluctant to publish replication studies that contradict the published literature. Fortunately, as Fetterman et al. [1] show in their scenario-based study, scientists appear to overestimate the negative reputational impact of a failed replication effort of their own work. In addition, admitting wrongness about a finding that cannot be replicated seems to be less harmful to one’s reputation than not admitting.
The scientific value of replication studies can be exemplified by a series of recent publications, some of which are highlighted in A Decade of Missing Pieces. Alongside many other groups [2,3,4], the replication studies by Javidi-Parisjani et al. [5] and Khin et al. [6] showed that the reported genome editing activity of NgAgo [7] was not reproducible. These replication studies refuting the original observations shortly after the publication of the original study helped clarify the limitations of the technique and correct the scientific literature. Another example of replication studies which had an impact include those refuting the findings of Wakefield et al. 1998 [8] which wrongly suggested a link between autism and vaccination. One of the studies highlighted in this collection, by Hornig et al. [9] was one of many that helped resolve this long-standing controversy.
Time has shown the value of publishing negative results and replication studies, and scientists are increasingly reporting these. However, for this cultural shift to continue, researchers’ concerns about publishing null and negative results must be overcome. Funding agencies can play a role by recognising the value of these studies that they ultimately fund, and journals can help by providing a venue for the publication of all rigorously conducted science, regardless of outcome. This collection A Decade of Missing Pieces highlights some of those studies published in PLOS ONE that have benefited the community by providing a different angle to a scientific story. By publishing this collection, we continue to applaud those that have published their missing pieces.
References:
- Fetterman A, Sassenberg K (2015) The Reputational Consequences of Failed Replications and Wrongness Admission among Scientists. PLOS ONE; https://doi.org/10.1371/journal.pone.0143723
- Burgess S, Cheng L, Gu F, Huang J, Huang Z, Lin S, Li J, Li W, Qin W, Sun Y, Songyang Z, Wei W, Wu Q, Wang H, Wang X, Xiong J, xi J, Yang, Zhou B, Zhang B. (2016) Questions about NgAgo. Protein & Cell; 7 (12) pp 913–915 https://doi.org/10.1007/s13238-016-0343-9
- Lee S, Turchiano G, Ata H, Nowsheen S, Romito M, Lou Z, Ryu S, Ekker S, Cathomen T, Kim J. (2017) Failure to detect DNA-guided genome editing using Natronobacterium gregoryi Argonaute. Nature Biotechnology; 35 pp17-18 https://doi.org/10.1038/nbt.3753
- Qi J, Dong Z, Shi Y, Wang X, Qin Y, Wang Y, Liu D (2016) NgAgo-based fabp11a gene knockdown causes eye developmental defects in zebrafish. Cell Research; 1349–1352 https://doi.org/10.1038/cr.2016.134
- Javidi-Parsijani P, Niu G, Davis M, Lu P, Atala A, Lu B (2017) No evidence of genome editing activity from Natronobacterium gregoryi Argonaute (NgAgo) in human cells. PLOS ONE https://doi.org/10.1371/journal.pone.0177444
- Khin N, Lowe J, Jensen L, Burgio G (2017) No evidence for genome editing in mouse zygotes and HEK293T human cell line using the DNA-guided Natronobacterium gregoryi Argonaute (NgAgo) PLOS ONE; https://doi.org/10.1371/journal.pone.0178768
- Gao F, Shen X, Jiang F, Wu Y, Han C (2016) DNA-guided genome editing using the Natronobacterium gregoryi Argonaute Nature Biotechnology; 34, pp768–773
RETRACTION: DNA-guided genome editing using the Natronobacterium gregoryi Argonaute Nature Biotechnology 2017; 35 pp797 https://doi.org/10.1038/nbt0817-797a - Wakefield AJ1, Murch SH, Anthony A, Linnell J, Casson DM, Malik M, Berelowitz M, Dhillon AP, Thomson MA, Harvey P, Valentine A, Davies SE, Walker-Smith JA (1998). Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet;351(9103), pp637-41.
RETRACTION: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children (2010) Lancet; 375(9713), pp445 https://doi.org/10.1016/S0140-6736(10)60175-4 - Hornig M, Briese T, Buie T, Bauman M, Lauwers G, Siemetzki U, Hummel K, Rota P, Bellini W, O’Leary J, Sheils O, Alden E, Pickering L, Lipkin W (2008) Lack of Association between Measles Virus Vaccine and Autism with Enteropathy: A Case-Control Study. PLOS ONE https://doi.org/10.1371/journal.pone.0003140