Registered Reports: One Year at PLOS ONE

A little over a year ago, PLOS ONE launched two new submission formats: Registered Report Protocols, peer-reviewed articles that describe planned research not yet initiated, and their follow-up Registered Reports, which report the results of the completed research and which receive an in-principle acceptance when their protocol is accepted for publication. It was part of a broader push for preregistration at PLOS.

When we added these options to the list of regular submission types we consider, the format wasn’t new: about 200 journals had already considered registered reports for publication, and the number has kept increasing since. The format had initially been relatively welcomed in the behavioral sciences and then made its way, sometimes with a few tweaks, to other disciplines. Preregistration in general has even been the norm in clinical trial research for years, albeit not necessarily with peer-review. And Registered Reports weren’t even entirely new at PLOS ONE: our partnerships with the Children’s Tumor Foundation and FluLab predate this launch.

But this launch had two distinctive features: we would publish the protocol (also called “stage-1 registered report”) of all the registered reports we would consider. We would do so regardless of the eventual results of the planned research, of course, but also regardless of whether the final report (also called “stage-2 registered report”) would be submitted or even completed. The Registered Report Protocol would be its own publication, and it would be so with the standard of any PLOS publication: with our expectations of data availability and rigorous ethics oversight, and with the possibility to make the full peer-review history available. It was, as far as we were aware, a distinctively transparent publishing format offering.

Other journals already published stage-1 Registered Reports, to be sure, but not at that scale and with the disciplinary breadth that PLOS ONE provides. This was this launch’s second distinctive feature: we were relying on an academic board of thousands of members to embrace this format with a different review process and criteria on as many study types and topics as the journal would normally consider.

For the 1st time since @RegReports were created in 2013, there is now at least one journal option for every research field across the full spectrum of physical, life and social sciences.

Chris Chambers, on PLOS ONE launching Registered Reports

The Registered Report format has been adapted and implemented in many ways across hundreds of journals (for instance at PLOS Biology). We made some choices with our own format: although deviations from the published protocol could invalidate the in-principle acceptance of the final report, we would consider such deviations, provided they are acknowledged and justified. We would also welcome exploratory, unregistered, or unplanned analyses in the final report, provided they are clearly identified as such. A Registered Report Protocol is an opportunity to receive early feedback on a study; it is the opportunity to claim ownership of a research project without having to wait for results to come in; it is also a tool against publication bias that drives us all not to publish null results. Above all, we envisioned Registered Report Protocols as a a mechanism for transparency in publishing and reporting rather than an unbreakable and inflexible vow. 

Our choice to distinguish clearly between the protocol and its final report, on the other hand, makes our format less adaptable to serial submissions and iterative registrations (which other journals publishing Registered Reports explicitly welcome). But authors wishing to do so with PLOS ONE can submit subsequent iterations of their registration (i.e., after the first follow-up to a published Registered Report Protocol) as regular research articles. But with that caveat, we wanted a format that is relatively flexible and that could be suited to as many study types and fields as we normally consider.

So what can we say a year later? We have received over 300 Registered Report Protocol submissions, about 60 of which are already published or accepted for publication (the first Registered Report Protocol was published in June of last year), by first authors from more than 20 countries. These submissions have acceptance and rejection rates comparable to our regular submissions. They cover many disciplinary areas: about 70% of the submissions are in medicine and health sciences, 15% in the behavioral and social sciences, and 8% in the life sciences. A call for papers in cognitive psychology, launched last fall in collaboration with the Center for Open Science, invited Registered Report Protocol submissions. Finally, we have already received a few follow-up Registered Report submissions. If and when we publish these stage-2 Registered Reports, they will be interlinked with their corresponding protocols so readers can easily navigate between them. 

The Registered Report Protocol submissions we received this past year are now published protocols for a systematic review on the effectiveness of public health interventions against COVID-19, a psychology survey study on trust in international relations, a study on neural plasticity, a study of biomedical sentence similarity measures, an animal and human study on the long-term effects of early-life stress, among others. They have been handled by a number of our Academic Editors and reviewers, many of whom were just discovering that Registered Reports were an option in their field. The journal’s editorial board members and reviewers have been instrumental in this successful rollout. As Andrew Miles, author of a published Registered Report Protocol, attested, “my research team and I benefited from careful reading by several excellent reviewers, as from an editor who pointed us to a data collection tool that we hadn’t previously been aware of.” 

Reproducibility of medical research findings has been found to be low, and Registered Reports give me the unique opportunity to describe in detail the statistical-methodological approach prior to having seen the data, and to get credit for it with respect to visibility in authorship. When we submitted our registered report to PLOS ONE we received very detailed reviewer comments, and we could improve our study design and analysis, as well as reporting. PLOS ONE publishes the [Registered Report Protocol] prior to the final study results, which has the advantage that the study can be brought to other people’s attention at a much earlier stage.

Ulrike Held, PLOS ONE Author
Is reporting quality in medical publications associated with biostatisticians as co-authors? A registered report protocol

Registered Reports are now just one of an increasing menu of publication formats. Recently, PLOS ONE launched new protocol types: Study Protocols and Lab Protocols. The Study Protocol format closely resembles that of Registered Report Protocols, but doesn’t come with an in-principle acceptance of the final report. Under the leadership of our new Editor-in-Chief Emily Chenette, PLOS ONE will continue to work with our communities to improve scientific communication, using the principles of openness, transparency, rigor, and reproducibility as guides.