Registered Reports with secondary developmental data: Introduction to the special issue

Events of the past decade have revealed substantial limitations in our standard approach to evaluating manuscripts for publication. Preference for 'positive results' and findings that are surprising or novel has led to a substantial publication bias that casts doubt on large portions of the existing literature (Davis-Kean & Ellis, 2019; Scheel et al., 2021). Registered Reports represent a major initiative to combat these problems, as they shift the focus of evaluation from the nature of the findings to the strength of the conceptualization, research design and analytic plan (Chambers, 2013). Contrary to the standard review process, with Registered Reports the process is split into two distinct stages. Authors initially submit a Stage 1 proposal consisting of the Introduction, Method and Planned Analysis sections prior to conducting the study. The Stage 1 proposal is sent for peer review, with an ultimate positive outcome of an 'in principle acceptance,' which is a guarantee that the journal will publish the full article, regardless of the results, providing the authors conduct the study as planned and do so competently. Following the in principle acceptance authors collect the data and/or conduct the analysis and then submit the Stage 2 report for final review. For more information, and answers to frequently asked questions about Registered Reports, see https://cos.io/rr/. Registered Reports have been increasingly adopted in journals across the sciences in general, and psychology in particular. Although uptake had initially been slow amongst developmental journals, this has changed considerably in recent years (see Syed et al., 2023, for Registered Reports specifically and Silverstein et al., 2024, for open science and metascience more generally). Nevertheless, there remain many questions about how the format works for complex longitudinal designs and secondary data (van den Akker et al., 2021), both of which are common in developmental research (see also Syed & Donnellan, 2020). The purpose of this Special Issue was to feature Registered Reports using secondary (pre-existing) data pertaining to developmental issues from the prenatal period through early adulthood. Secondary datasets refer to data collected by someone other than the primary user. Datasets used in the eight articles featured in the special issue covered wide ground. Two of the articles relied on the National Institute of Child Health and Human Development Study of Early Child Care and Youth Development, one to investigate how early caregiver interactions are related to educational attainment, income and employment (Duncan et al., 2024), and the other to compare relations between parental sensitivity and two different methods for assessing attachment (Nivison et al., 2024). Examining parenting from a different perspective, Wright and Jackson (2024) drew from the German Socioeconomic Panel study to test the independent predictive power of parent and child personality traits on a variety of adolescent outcomes (e.g., health, education, family and civic engagement). Several studies focused on the schooling context: Spiegler et al. (2024) used the Children of Immigrants Longitudinal Survey in Four European Countries to examine how peer victimisation was related to classroom ethnic diversity and teacher interactions; Kim and Sidney (2024) used the Trends in International Mathematics and Science Study to examine sources of influences for individual differences in students' academic self-concept; and Zimmermann et al. (2024) used the Early Childhood Longitudinal Study Kindergarten class of 2011 to understand how pre-Kindergarten enrollment was related to indicators of social development across the transition into and out of Kindergarten. Finally, two studies examined physiological aspects of development: Chaku and Barry (2024) used the Adolescent Brain Cognitive Development study to understand how hormonal profiles were associated with cognitive development, and Woods et al. (2024) drew from the Future of Families and Child Wellbeing Study to test whether indicators of sleep were longitudinally associated with cognitive and behavioural development. Taken together, the articles in the current Special Issue highlight both the feasibility and utility of Registered Reports with secondary developmental data. It is abundantly clear from the papers published for this special issue that Registered Reports are a viable option for those who are using existing or secondary datasets, and that they only scratch the surface of available datasets that could be put to good use with Registered Reports. Some other examples of secondary datasets include but are not limited to Monitoring the Future, Panel Study of Income Dynamics, National Longitudinal Surveys, British Cohort Studies, Growing Up in Scotland and many more (see Davis-Kean & Ellis, 2019, for a review, and Davis-Kean et al., 2015, for more details on how to access these and other datasets). All of these datasets, and many more, are potentially suitable for Registered Reports but are underused with the format. We do, however, want to highlight a few important issues that were noted during the review process. First, reviewers sometimes struggle with how to review papers that do not contain data, but only the 'proposal' of the idea and methods. For secondary data in particular, reviewers struggled with the limitation of existing data already having been collected and not being able to advise on new primary data collection. Often, as has been noted by Davis-Kean et al. (2015), there are limitations on what questions can be answered with secondary data. There is no option to add data to the existing data or change the methods of the study. Reviewers would often suggest such changes for these papers that were not possible. Thus, a recommendation for future secondary data Registered Reports is that very clear reviewer information be provided on the role of the reviewer but also the limitations of what can be requested by the reviewer to the authors. Basically, the methodology and data structure are generally already known for the study, the reviewers' job is to determine if the questions are of interest to the field of study and if the methods and the data set being used are suitable to answer the questions being asked. A challenge that exists in the interaction between the reviewers and authors, evident for both preregistrations and Registered Reports, is the frequent need for robustness checks that may not have been previously registered, but are important for validating the findings. These are often seen as exploratory analyses but often are reasonable requests for checking the validity of a finding when a reviewer notes an alternative hypothesis that is both plausible and testable in the data set. Unlike primary data collection which may be restricted in the breadth of variables available to do the robustness check, secondary data will often have the available constructs that could be tested. We recommend authors and reviewers think through the addition of alternative specifications or hypotheses that could be tested with the existing data at the Stage 1 proposal phase. Sometimes, however, such realisations occur after the fact and thus must be added as exploratory analyses to the Stage 2 manuscript. Authors are encouraged to do this, and editors and reviewers are encouraged to facilitate it, so long as the additions are transparently reported, as we always want to be sure we are conducting the most rigorous tests in our studies. Finally, on the initial submission for review for this special issue, it was sometimes the case that the actual questions being submitted were not major questions of the field, or could not adequately be tested with the proposed dataset. Any analysis of secondary data could be done, but it is important for us to all consider whether it should be done. One of the primary goals of Registered Reports is to ensure that a study is informative of its research questions regardless of the results. This design feature demands high-quality studies and is particularly well-suited for the most complex, controversial and transformative research because it provides the clearest constraints on the questionable research practices and publication bias that often plague developmental research (Davis-Kean & Ellis, 2019). Elevating this format to being one where having authorship on a Registered Report denotes that extreme care and careful planning were done to obtain the finding will help to increase the confidence in the research that is done on infant and child development.