Reproducibility Practices in Scientific Computing: A Community Survey

Reproducibility is a cornerstone of scientific practice, enabling researchers to verify findings and adapt methods for their own investigations. The importance of reproducibility has prompted several surveys within the scientific community to assess its current state. Notably, a widely recognized study conducted by Nature revealed that more than half of responding researchers acknowledged the existence of a significant reproducibility crisis 1.Recently, systematic replication studies — particularly in cancer biology — have identified and quantified specific factors rendering published experimental results not reproducible 2. While these studies have illuminated reproducibility challenges mainly in traditional experimental sciences, we hypothesize lack of computational reproducibility and hence transparency also poses significant problems in computational sciences, including bioinformatics, astrophysics, computational chemistry, and many more computation-reliant fields, due to the interdisciplinary nature of computing.To assess community needs and identify gaps that must be addressed to improve current practices, we conducted an online survey (N = 422) among users of national and institutional HPC and cloud resources. The questionnaire explored:- computational reproducibility awareness and implemented measures,- interest in Workflow Management Systems (WMS) to formalize and automate complex analyses,- current adoption patterns and preferred tools (e.g., Snakemake, Nextflow), and - perceived barriers to reproducible, transparent data and software management. The survey design allowed us to capture perspectives across different computational disciplines, providing insights into how reproducibility awareness and practices vary among research communities. Preliminary findings indicate strong interest in WMS, though only a fraction of respondents report regular use. Additionally, concerns about data provenance, version control, and long‑term archiving emerged as recurring themes. This poster will present the survey design, key statistics, and an early discussion of how HPC centers and software developers might help reduce these obstacles, fostering more reproducible scientific workflows. Results are preliminary and will be refined for a forthcoming peer‑reviewed publication. 1 Baker, M. 1,500 scientists lift the lid on reproducibility. Nature 533, 452–454 (2016). https://doi.org/10.1038/533452a2 Errington, T. M., Mathur, M., Soderberg, C. K., Denis, A., Perfito, N., Iorns, E., & Nosek, B. A. Investigating the replicability of preclinical cancer biology. eLife 10, e71601 (2021). https://doi.org/10.7554/eLife.71601