With Registered Reports (RRs), academic journals review the research question and study design before data is collected. At this stage, they already decide whether the study will be published, regardless of its later results. In this way, RRs – which are becoming increasingly common in the field of economics and business research – help to counteract questionable publication and research practices. In this article, you will learn what an RR is suitable for, where it can be submitted and how to make it a success.
Many questionable research practices share a common cause: the incentive structure of the scientific publishing system. Studies with statistically significant results are more likely to be published (publication bias) and more frequently cited (citation bias) than studies with non-significant results. At the same time, researchers are under pressure to publish. This creates incentives for questionable practices: for example, adjusting analysis methods until a significant result appears (so-called p-hacking), or presenting hypotheses formed after the fact as if they had existed before data collection (so-called HARKing).
RRs break this incentive structure. Researchers submit a detailed study plan to a journal before carrying out the study. Journals review this plan and can then grant an “In-Principle Acceptance” (IPA). This guarantees publication, regardless of the study’s outcome, as long as the plan is followed. What an RR is exactly, we explain in more detail elsewhere. Here we'll look at the practical implementation.
What is an RR suited for?
RRs are suited, first, to analyses in which researchers collect the data themselves – for example, laboratory and field experiments, Randomised Controlled Trials (RCTs), or survey experiments. Such studies are typical of experimental economics as well as of marketing, HR, or behavioural studies in business research. RRs are also possible for secondary data analyses using data that already exist, provided researchers disclose the extent to which they had already seen the data before review. Meta-analyses, systematic reviews, and qualitative studies with pre-specified research questions can likewise be submitted as an RR.
Where can I submit my RR?
There are two routes: submitting directly to a single journal, or submitting via the journal-independent platform PCI-RR. For the first option, you should check the specific requirements of the journal in question and which study types it accepts. The COS maintains a complete, continuously updated list. Arpinon’s list complements this with journals that accept RRs in economics specifically. PCI-RR reviews almost any type of RR free of charge and across disciplines. Following a positive review, you automatically receive In-Principle Acceptance from all participating “PCI-RR-friendly” journals.
What makes an RR succeed?
The following five tips draw heavily on the ten general rules by Henderson and Chambers (2022) for writing an RR.
- Formulate a precise, testable research question
Formulate a research question in the introduction of your plan that can be tested with the data you plan to collect and a sound, feasible method. The question should remain interesting even if the hypotheses are not confirmed. Your hypotheses should follow directly from the question, be stated precisely, and translate into measurable quantities.
- Say what you will do – and what you won’t
Your plan should be specific and complete enough that someone outside your field could carry out your study based on the description alone, without needing to ask further questions. To achieve this, specify not only what you will do, but also what you will not do. For example: “If the software crashes during a session, we will exclude the affected observation and will not repeat the session.”Your plan should be specific and complete enough that someone outside your field could carry out your study based on the description alone, without needing to ask further questions. To achieve this, specify not only what you will do, but also what you will not do. For example: “If the software crashes during a session, we will exclude the affected observation and will not repeat the session.”
- Link your question, analysis, and interpretation transparently
Make sure that every hypothesis from your introduction is clearly connected to the corresponding analysis and the interpretation derived from it. A design summary table is well suited for this, setting out the research question, sampling plan, analysis, and expected results side by side. Henderson and Chambers (2022) provide a template as well as examples of completed tables (see the appendices of their article). If your subsequent course of action depends on the outcome of an intermediate step, set out if-then rules in advance, for example: “If most respondents give similar answers (low variance), we will use the median rather than the mean, since the latter is easily distorted by outliers when variance is low.”
- Work out your analysis plan carefully
The analysis plan is the heart of an RR. It sets out in advance how the data will be analysed. For quantitative, confirmatory studies – that is, studies that test a pre-specified hypothesis – it covers, for example, the choice of statistical model or the sample size determined in advance. Arpinon and Espinosa (2023) explain in detail how to work out these elements and provide ready-to-use R and Stata code for typical behavioural economics experiments such as public-goods or dictator games.
- Document any deviations
Unforeseen events, such as a technical error, may require a subsequent adjustment to your plan. For major changes (for example, to the analysis method), seek the editor’s approval in advance. If in doubt, it is better to ask than to risk rejection due to an unauthorised deviation. Document every change transparently. The same applies to anything that was not pre-specified: additional findings may be reported, provided they are clearly labelled as exploratory.
An RR may sound like more work at first glance. In practice, however, it merely shifts the effort: more planning upfront, but in return a publication guarantee independent of the outcome, and early feedback instead of rejection after the costly process of data collection. As Henderson and Chambers (2022) note, this can also take away researchers' fear of "wrong" results and thereby spare them a lot of stress.
Tip: Once your study is complete and the final manuscript has been accepted, you should make your original study plan, along with your data, code, and materials, openly available in a replication package. What matters when it comes to the accompanying README file is explained in this blog post on the Open Economics Guide.