Quantifying tensions in cosmological parameters: Interpreting the DES evidence ratio

TL;DR

This work reframes the DES Y1 tension with Planck by scrutinizing the Bayes ratio $R$ and its prior dependence, arguing that $R$ should be interpreted as Bayesian confidence in combining datasets rather than a model-compatibility metric. It introduces a prior-insensitive tension measure via the information ratio $\mathcal{I}$ and suspiciousness $\mathcal{S} = \log R - \log I$, grounded in Kullback-Leibler divergences, and adds Bayesian model dimensionality to quantify how many parameters are effectively constrained. Through analytic (top-hat and Gaussian) examples and a detailed cosmological application (Planck, DES Y1, SH0ES, BOSS), the paper finds moderate tension between DES and Planck under reasonable priors and strong tension between SH0ES and Planck, while BAO+RSD remains consistent with Planck. It also demonstrates that the combination $\log \mathcal{Z}}+\mathcal{D}$ is relatively prior-stable, and introduces a practical framework for dataset comparison ahead of future DES releases and upcoming surveys.

Abstract

We provide a new interpretation for the Bayes factor combination used in the Dark Energy Survey (DES) first year analysis to quantify the tension between the DES and Planck datasets. The ratio quantifies a Bayesian confidence in our ability to combine the datasets. This interpretation is prior-dependent, with wider prior widths boosting the confidence. We therefore propose that if there are any reasonable priors which reduce the confidence to below unity, then we cannot assert that the datasets are compatible. Computing the evidence ratios for the DES first year analysis and Planck, given that narrower priors drop the confidence to below unity, we conclude that DES and Planck are, in a Bayesian sense, incompatible under LCDM. Additionally we compute ratios which confirm the consensus that measurements of the acoustic scale by the Baryon Oscillation Spectroscopic Survey (SDSS) are compatible with Planck, whilst direct measurements of the acceleration rate of the Universe by the SHOES collaboration are not. We propose a modification to the Bayes ratio which removes the prior dependency using Kullback-Leibler divergences, and using this statistical test find Planck in strong tension with SHOES, in moderate tension with DES, and in no tension with SDSS. We propose this statistic as the optimal way to compare datasets, ahead of the next DES data releases, as well as future surveys. Finally, as an element of these calculations, we introduce in a cosmological setting the Bayesian model dimensionality, which is a parameterisation-independent measure of the number of parameters that a given dataset constrains.

Figures (9)

• Figure 1: Tension between the S$H_0$ES and Planck datasets as exhibited by examining the posterior parameter constraints on the Hubble constant.
• Figure 2: No tension between BOSS and Planck datasets as exhibited by examining the joint posterior parameter constraints on the matter fraction and $\sigma_8$.
• Figure 3: Possible tension between DES and Planck datasets as exhibited by examining the joint posterior parameter constraints on the matter fraction and the parameter combination $S_8 = \sigma_8 (\Omega_m/0.3)^{0.5}$.
• Figure 4: Many non-Gaussian posteriors (left) may be "Gaussianised" (right) by using Box-Cox transformations.
• Figure 5: Log-evidence $\log \mathcal{Z}$ and Kullback-Leibler divergence $\mathcal{D}$ calculations for all datasets and priors considered in this paper. The figures show the numerical values for the log-evidence and Kullback-Leibler divergence for the likelihoods described in \ref{['sec:datasets']} under the default and narrow priors summarised in \ref{['fig:prior']}, with red representing results for the default priors, orange medium priors and blue narrow priors. One can see that narrowing the prior increases the log-evidence and reduces the Kullback-Leibler divergence, but that $\log \mathcal{Z} +\mathcal{D}$ remains constant to within error. It should also be noted that the errors in estimating $\log \mathcal{Z}$ and $\mathcal{D}$ are strongly correlated. These errors arise from the uncertainty inherent in nested sampling's estimate of the volume compression of each likelihood contour, and influences both quantities in the same manner. It should be noted that the parameter combination that we are most interested in estimating ($\log \mathcal{Z} + \mathcal{D}$) has the lowest error in its estimation.

Theorems & Definitions (2)

• Proposition 1
• Proposition 2