Mapping Open Science: New Data on Pre-Registration in Experimental Economics
Imai, Toussaert and co-authors: "Pre-Registration and Pre-Analysis Plans in Experimental Economics" CRC Discussion Paper No. 530
The integration of open science practices, particularly pre-registration and the use of pre-analysis plans, has reshaped ongoing debates about research credibility in the social sciences. A recent study by Taisuke Imai (University of Osaka, associated with Project B01) and Séverine Toussaert (Oxford University), and co-authors (among them Levent Neyse, DIW, inf Project) offers concrete evidence of the adoption and evolving perspectives on this practice specifically within experimental economics.
Combining longitudinal observational data with a detailed survey, the authors show that pre-registration has become a mainstream practice in the field, even though researchers continue to debate how extensive and detailed these plans should be.
Examining published papers across 19 leading journals between 2017 and 2023, the study finds a substantial increase in adoption, with the number of papers featuring pre-registered experimental studies growing from seven per year to 190 per year. This significant growth suggests a broad institutional shift.
Survey responses from 519 researchers in experimental economics paint a similar picture: 86% of respondents reported having pre-registered at least one study, and two-thirds expressed a favorable view of the practice overall. The most common motivations were to signal credibility and to meet journal requirements. Importantly, the study also finds that researchers assign a “credibility premium” to statistically significant results derived from pre-registered hypothesis tests, linking the practice directly to enhanced perceived transparency and replicability.
Still, opinions remain divided over how pre-registration should be implemented. Most respondents agreed that all kinds of experiments, including lab, field, and online studies, should be pre-registered, but they disagreed about how comprehensive these plans need to be. A slight majority favored flexibility, saying that deviations from pre-registration are acceptable as long as they are clearly reported and justified.
Taken together, the findings point to a community in transition: pre-registration is now widely accepted, but standards for how it should be practiced are still evolving. The study highlights that there is a growing need for professional associations to provide structured guidelines for both authors and reviewers to navigate these developing open science norms effectively.
Link (pdf): Pre-Registration and Pre-Analysis Plans in Experimental Economics
