Global sensitivity analysis unveils the hidden universe of uncertainty in multiverse studies

We uncover how the “hidden uncertainty" seen in multi-analyst studies can be tackled using bestpractices in global sensitivity analysis (GSA). GSA allows the discovery of where the missing outputuncertainty is located and of why it has escaped previous analyses. GSA is superior to the existingmultiverse analysis in that (i) uses state of the art sampling strategies to explore the space of themodelling options – the so-called garden of the forking paths – and (ii) reveals how no single variablein isolation is responsible for a sizeable fraction of the output variance, which is explained instead byinteractions effects (second order and higher) among variables. GSA provides a chart for the analystsbefore they embark in their experiment and helps to understand its results after it is done. We makeour case by replicating via computer simulation the study by Breznau et al., which involved 161researchers in 73 teams and was published in PNAS in 2022.