The “responsible conduct of research” is not limited to properly obtained consent
Today we published the article,Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results, by Wicherts et al. In the manuscript, the authors show that weaker evidence in a psychology paper published in a journal requiring signed agreements to share data is associated with a failure to comply with this signed agreement to share.
In the following opinion piece, Rochelle Tractenberg, the academic editor who handled the peer review of Wicherts et al., discusses the ways in which “research ethics” and the “responsible conduct of research” go beyond human subjects protection, to include data sharing, professional conduct, and the careful, correct and complete reporting of all analyses in published research. Rochelle is a biostatistician and research methodologist at Georgetown University.
The article by Wicherts et al. describes their study of 1148 “statistically significant results” that were published in 49 different papers within two psychology journals. As it was being reviewed and revised, this paper sparked a vigorous debate among PLoS ONE editors and reviewers for two very different, ethics-oriented, reasons. These reasons were:
- Disciplines differ in their requirements that data analyzed and published should be made available to any scientist that requests the data. In some cases, the data cannot be properly de-identified, which violates a key feature of human subjects protections that all scientists are supposed to commit to whenever they carry out scientific work involving living persons. The Ethics Committee of the authors’ institution explicitly required them NOT to publicize the authors who refused to share data – and whose work was identified as being weak in terms of evidence and/or quality of results.
- The readers of published work are totally dependent on the quality and accuracy of the statistical results presented –and described – in any manuscript they wish to utilize as evidence in their own argument or scientific endeavor. The option to re-analyze data is critical to check the reliability of published designs, results, or conclusions. Time is a precious commodity; knowing what work is most likely to require replication prior to its use would limit our reliance on work likely to include mis-analyzed data, misinterpreted results, or both.
These considerations highlight diverse conceptualizations of the “responsible conduct of research”: in the first conceptualization, the protection of human subjects –the authors who did and did not share their data, whose published work was analyzed in this paper – is the sole concern. The second conceptualization involves the responsibility of the authors of the reviewed published papers to fulfill their professional ethical obligations as well as their contractual agreements. Wicherts et al. did obtain permission from their Ethics Committee to provide a list of authors who failed to share data – if specifically requested, but are NOT permitted to identify which authors were compliant and defiant within their paper, nor to place their data file, as is, into a public archive. Wicherts et al. analyzed the contents of published articles, and then requested the data these papers represented. The journals’ and APA ethics policies are public and require data sharing, so neither the requests nor the failures to comply with those requests can be considered privileged. In short, no feature of the data in this manuscript can be considered to be privileged. The responsible conduct of research is much more than protecting study subjects and obtaining informed consent, but the Ethics Committee did not act like it is.
As the editor shepherding this article through the review process, I applaud the effort by Wicherts and colleagues, and disagree with the Ethics Committee decision. I believe that every journal should document requests for data and the failures of authors to provide the requested data as part of the publication record. This compliance information should be accessible to all reviewers for journals and grants, and to scientists who look to the published literature for replicable, reliable, ethically obtained results.
A prior report by Wicherts et al. in 2006 found that, while journals published by the APA all require authors to sign an agreement to share their data, 73% of authors who were contacted for that study failed to do so. These 73% defied their written agreement, and professional obligation (see APA Ethical principle 8.05), to share their data. This is not responsible conduct of research; this is unethical behavior and every scientist is entitled to know whether or not an author is responsible in their conduct –and reporting – of their research. As critical as these features are to responsible conduct of research, the review of manuscripts and grants for “ethical considerations” cannot be limited to the inclusion of women and minorities, balance between male and female subjects, and properly obtaining informed consent.
Wicherts et al. conclude that policies for data archiving should be strengthened/more widespread, even though their 2006 findings that, in defiance of their signed agreements, the journals’ publishing policy, and the professional association policy on data sharing, nearly ¾ of APA published authors refused to share data when contacted. In their current study, the refusal rate was “only” 57%.
A survey of geneticists working at 100 universities in the US led to an estimate of 47% failure to share. The challenge is clearly widespread and has a prevalence rate in the 47-73% range. By contrast, in her survey of the prevalence of “unethical” or “questionable” scientific behavior, Fanelli arrived at far smaller estimates – possibly because of a lack of awareness among her respondents that a failure to share data or research resources constitutes irresponsible –not merely questionable – research practice.
A broader appreciation of how critical transparency and replicability are for good science could support the cultural shift necessary to promote the idea that the “responsible conduct of research” is not limited to properly obtained consent. A mechanism for documenting failures in these key features of responsible conduct of research would also support this shift. Every reviewer should have this information about the likelihood that an applicant or author will violate sharing policies and/or defy contracts they’ve signed and/or professional ethical science standards they should be held to.
The responsible conduct of research includes conscientious protection of human subjects. In the book, “Scientific Integrity”, Macrina also notes, “(s)haring research materials published in the peer reviewed literature has been a traditional practice that follows from the expectation that scientific research must be amenable to replication” (p. 81) and that “…written communication is an essential part of scientific research.” (p. 83) (see also National Research Council, 1992; Board on Life Sciences, 2003). As such, these should be just as conscientiously executed as other aspects of the research enterprise. Refusals to share data harms science and constitutes violations of our obligations to be responsible conductors of research.
Rochelle Tractenberg has also written a companion piece to this article, which can be found in the comments section of the Wicherts et al. manuscript. In the second piece, she interprets the association between weak evidence and failures to comply with data sharing requirements in a different way. Instead of supporting mandatory archiving of data, Dr. Tractenberg concludes that readers and reviewers of manuscripts and grant proposals should be notified of the author’s/applicant’s history of compliance with data sharing mandates and policies. She believes that such documentation will do more to change the culture towards one promoting science and data sharing than mandatory data archiving. To read this post, please click here.
References
Board on Life Sciences (2003). Sharing Publication-Related Data and
Materials: Responsibilities of Authorship in the Life Sciences.
Washington, DC: National Academy Press.
Campbell, E. G., Clarridge, B. R., Gokhale, M., Birenbaum, L.,
Hilgartner, S., Holtzman, N. A., et al. (2002). Data withholding in
academic genetics: Evidence from a national survey. Journal of the
American Medical Association, 287, 473–480.
Fanelli D (2009) How Many Scientists Fabricate and Falsify Research? A
Systematic Review and Meta-Analysis of Survey Data. PLoS ONE 4(5):
e5738. doi:10.1371/journal.pone.0005738
Macrina FL. (2005). Scientific Integrity: Text and Cases in
Responsible Conduct of Research, 3E. Washington, DC: American Society
for Microbiology Press.
National Research Council (1992). Responsible Science, Vol 1: Ensuring
the Integrity of the Research Process. Washington, DC: National
Academy Press.
Wicherts JM, Borsboom D, Kats J, Molenaar D (2006) The poor
availability of psychological research data for reanalysis. American
Psychologist 61:726-728.
Wicherts JM, Bakker M, Molenaar D (2011) Willingness to Share Research Data Is Related to the Strength of the Evidence and the Quality of Reporting of Statistical Results. PLoS ONE 6(11): e26828. doi: http://dx.plos.org/10.1371/journal.pone.0026828