Conservative cosmology: combining data with allowance for unknown systematics

TL;DR

The paper tackles the problem of combining cosmological data when unknown systematics threaten reliability. It introduces BACCUS, a Bayesian hierarchical framework that uses shift parameters per experiment class and a flexible covariance prior to yield conservative, tension-robust parameter posteriors. Through simple illustrative examples and a cosmological application to the H0 tension, the method demonstrates fat-tailed posteriors and automatic downweighting of discordant data, while remaining informative when data are concordant. The approach provides a pragmatic path toward credible inferences for next-generation surveys where unknown systematics may dominate statistical errors.

Abstract

When combining data sets to perform parameter inference, the results will be unreliable if there are unknown systematics in data or models. Here we introduce a flexible methodology, BACCUS: BAyesian Conservative Constraints and Unknown Systematics, which deals in a conservative way with the problem of data combination, for any degree of tension between experiments. We introduce hyperparameters that describe a bias in each model parameter for each class of experiments. A conservative posterior for the model parameters is then obtained by marginalization both over these unknown shifts and over the width of their prior. We contrast this approach with an existing hyperparameter method in which each individual likelihood is scaled, comparing the performance of each approach and their combination in application to some idealized models. Using only these rescaling hyperparameters is not a suitable approach for the current observational situation, in which internal null tests of the errors are passed, and yet different experiments prefer models that are in poor agreement. The possible existence of large shift systematics cannot be constrained with a small number of data sets, leading to extended tails on the conservative posterior distributions. We illustrate our method with the case of the $H_0$ tension between results from the cosmic distance ladder and physical measurements that rely on the standard cosmological model.

Figures (9)

• Figure 1: Comparison of the results obtained using shift parameters and the conventional approach to combining data sets in a model with only one parameter, $a$, and $N$ data sets whose individual posteriors are Gaussians. We show individual posteriors in black, the posteriors obtained with the conventional approach in orange and the posterior obtained with our approach, in purple. The dependence of the posterior on the number of data sets for the exactly consistent case is shown in the top panels, while strongly inconsistent cases are shown in the bottom panels.
• Figure 2: The same as Figure \ref{['fig:1D_2-4']}, but considering cases in which all the data sets are consistent (panels $a$ and $b$), only one is discrepant with the rest (panels $c$ and $d$), eight data sets with scatter larger than the errors (panel $e$) and eight data sets with random values of the best fit and errors (panel $f$).
• Figure 3: Constraints for six data sets sampled from a straight line with slope $m=1$ and intercept $c=1$ ($\lbrace D_1\rbrace$ and $\lbrace D_2\rbrace$). We show the individual posteriors in black and the results from using the conventional approach in orange, using rescaling parameters, in blue, using shift parameters, in purple, and using both in green. Top left: all data sets have 50 points. Top right: all data sets have 5 points. Bottom panels: as in the top panels, but the errors of $\lbrace D_2\rbrace$ are underestimated a factor 5.
• Figure 4: As Figure \ref{['fig:comp_cons']} but using $\lbrace D_3\rbrace$ (with slope $m=0$ and intercept $c = 1.5$) instead of $\lbrace D_2\rbrace$.
• Figure 5: As Figure \ref{['fig:comp_cons']} but using $\lbrace D_4\rbrace$ (with slope $m=0.7$ and intercept $c = 0.7$) instead of $\lbrace D_2\rbrace$.