A simple tool for weighted averaging of inconsistent data sets

TL;DR

This work addresses the challenge of combining inconsistent measurements by replacing the standard inverse-variance weighted average with a Bayesian framework that marginalises over unknown true uncertainties. Building on Sivia and Skilling, it introduces two priors—the conservative prior and the Jeffreys' prior—yielding non-Gaussian likelihoods with heavy tails that resist outliers and underestimation of uncertainties. The approach is demonstrated on synthetic data, CODATA values for the Newtonian constant, and PDG particle properties, showing generally more robust and realistic uncertainty estimates and revealing when full posterior information must be used instead of a single weighted mean. A freely available Python tool, bayesian_average, facilitates practical adoption, offering transparent comparisons with traditional methods and visualisation of the complete posterior distributions. This method provides a simple, broadly applicable alternative for robust data fusion in contexts where interlaboratory data and outliers distort standard analyses.

Abstract

The weighted average of inconsistent data is a common and tedious problem that many scientists have encountered. The standard weighted average is not recommended for these cases, and various alternative methods have been proposed. These approaches vary in suitability depending on the nature of the data, which can make selecting the appropriate method difficult without expertise in metrology or statistics. For the analysis of simple data sets presenting inconsistencies, we discuss the method proposed by Sivia in 1996 based on Bayesian statistics. This choice has the intention of maintaining generality while minimising the number of assumptions. In this approach, the uncertainty associated with each input value is considered to be just a lower bound of the true unknown uncertainty. The resulting likelihood function is no longer Gaussian but has smoothly decreasing wings, which allows for a better treatment of scattered data and outliers. To demonstrate the robustness and the generality of the method, we apply it to a series of critical data sets: simulations, CODATA recommended values of the Newtonian gravitational constant, and some particle properties from the Particle Data Group, including the proton charge radius. A freely available Python library is also provided for a simple implementation of the proposed averaging method.

Figures (8)

• Figure 1: Comparison between the different assumed probability distribution for each datum for $\mu=0,\sigma_i=1$.
• Figure 2: Standard and Jeffreys' weighted averages of different simulated data sets: data randomly sampled from a normal distribution (top) and with the addition of a random bias (middle) or an outlier (bottom).
• Figure 3: Final likelihood distributions for the data set 3 together with the input values (black). The solid lines represent the standard (red), conservative (green) and Jeffreys' prior likelihood distribution (blue). The corresponding average and standard deviation are marked by vertical lines.
• Figure 4: Comparison between the official CODATA values CODATA1969CODATA1973CODATA1986CODATA1998CODATA2002CODATA2006CODATA2010CODATA2014CODATA2018 of the Newtonian constant and the values obtained by the Bayesian weighted average using Jeffreys' prior. CODATA values obtained from single measurements are presented alone, as no weighted average could be performed. The small error bar of the CODATA values indicates the uncertainty calculated by the standard weighted average, and the large one indicates the recommended uncertainty. The horizontal dashed line corresponds to the latest CODATA value (2022 edition CODATA2022, equal to the 2018 edition value).
• Figure 5: Final likelihood distribution (in log scale) of the measurements of the Newtonian constant included in the CODATA 1998 compilation CODATA1998. The CODATA 1998 recommended value is also reported (in grey), which differs from the standard weighted average for the considered measurements (in red).