Bayes, E-values and Testing

TL;DR

The paper addresses how to quantify randomness and statistical evidence by unifying Kolmogorov complexity, Shannon entropy, Bayes factors, E-values, and exchangeability testing through the lens of negative log marginal/predictive probabilities $-\ log P$. It surveys information-theoretic and martingale tools, showing that the marginal likelihood ratio serves as a central E-value linking Bayesian inference and game-theoretic probability, and that Sanov's theorem and KL divergence govern both large deviations and evidence growth. It introduces conformal e-prediction as a model-free approach to testing exchangeability and demonstrates how E-values enable anytime-valid sequential testing and safe aggregation of evidence. The work provides a cohesive framework underpinning sequential decision making, model checking, and interim analyses, with practical implications for adaptive clinical trials and online inference.

Abstract

This paper studies relationships between Kolmogorov complexity, Shannon entropy, Bayes factors, E-values, and exchangeability testing. The focus is on negative log marginal or predictive probabilities -- what I.J.~Good termed the ``weight of evidence'' -- as a common evidence statistic linking coding, prediction, and sequential testing. The paper reviews the relevant information-theoretic and martingale tools, and discusses exchangeability testing via conformal e-prediction.

Theorems & Definitions (11)

• Definition 1: Sufficiency
• Theorem 2: Shannon-McMillan-Breiman
• Theorem 3: Barron 1985
• Definition 4
• Definition 5
• Theorem 6: Ville's Inequality
• Proposition 7
• proof
• Definition 8
• Theorem 9: de Finetti