Investigating the reproducibility of the social and behavioural sciences

Published claims should be reproducible, yielding the same result when applying the same analysis to the same data. We assessed reproducibility in a stratified random sample of 600 papers published from 2009 to 2018 in 62 journals spanning the social and behavioural sciences. Authors of 144 (24.0% [95% CI 20.8 - 27.6%]) papers made data available to assess reproducibility, and for 38 others, we obtained source data to reconstruct the dataset. We assessed 143 of the 182 available datasets and found that 76.6 (53.6% [95% CI 45.8 - 60.7%]) papers were rated as precisely reproducible and 105.0 (73.5% [95% CI 66.4 - 80.0%]) papers as at least approximately reproducible (within 15% of the original effects or within .05 of original p-values) after inverse weighting each of the 551 claims by the number of claims per paper. We observed higher reproducibility for papers from Political Science and Economics than other fields, for more recent than older papers, and for papers from journals that required data sharing. Implementing measures to verify that research is reproducible is needed to support trustworthiness in the complex enterprise of knowledge production.