Quantifying tensions in cosmological parameters: Interpreting the DES evidence ratio

We provide a new interpretation for the Bayes factor combination used in the Dark Energy Survey (DES) first year analysis to quantify the tension between the DES and Planck datasets. The ratio quantifies a Bayesian confidence in our ability to combine the datasets. This interpretation is prior dependent, with wider prior widths boosting the confidence. We therefore propose that if there are any reasonable priors which reduce the confidence to below unity, then we cannot assert that the datasets are compatible. Computing the evidence ratios for the DES first year analysis and Planck, given that narrower priors drop the confidence to below unity, we conclude that DES and Planck are, in a Bayesian sense, incompatible under. Additionally we compute ratios which confirm the consensus that measurements of the acoustic scale by the Baryon Oscillation Spectroscopic Survey (BOSS) are compatible with Planck, while direct measurements of the acceleration rate of the Universe by the Supernovae and H₀ for the Equation of State of Dark Energy Collaboration (SH₀ES) are not. We propose a modification to the Bayes ratio which removes the prior dependency using Kullback-Leibler divergences, and using this statistical test we find Planck in strong tension with SH₀ES, in moderate tension with DES, and in no tension with BOSS. We propose this statistic as the optimal way to compare datasets, ahead of the next DES data releases, as well as future surveys. Finally, as an element of these calculations, we introduce in a cosmological setting the Bayesian model dimensionality, which is a parametrization-independent measure of the number of parameters that a given dataset constrains.