Psychological scientists are increasingly acknowledging the importance of transparency for research integrity. The present study examined one important facet of transparency: providing enough information about measures so that readers can evaluate aspects of construct validity. With a focus on social and personality psychology, we explored how often authors in one journal report a scale name, citation, example item, number of items, and reliability coefficient, as well as how often authors provide access to the study’s materials. We also investigated how measurement reporting practices have changed from 2010 to 2020, the decade encompassing the start of the “credibility revolution” in psychology. Across two samples, we coded 506 Social Psychological and Personality Science (SPPS) articles (N = 425 articles with at least one questionnaire measure; 1,198 questionnaire measures). Overall, ~31% of measures were reported with a name, ~53% a citation, ~66% an example item, ~76% the number of items, and ~78% of multi-item measures included some reliability information; approximately 22% of measures were single-item and 46% were ad hoc. We did not detect any apparent changes in the reporting practices examined from 2010 to 2020 in either sample, except for an increase in the availability of materials over time. Therefore, the replication crisis may have motivated increased access to studies’ materials in recent years but otherwise does not seem to be associated with more transparent reporting of measurement information for questionnaires in brief social and personality articles.
Lawson et al. (Fri,) studied this question.