Algorithmic randomness and the weak merging of computable probability measures
Simon M. Huttegger, Sean Walsh, Francesca Zaffora Blando
TL;DR
The paper develops a framework for merging randomness and shows that Martin-Löf randomness and Schnorr randomness for a computable ν on Cantor space can be characterized via weak merging with respect to KL-divergence increments. By relating the KL-divergence D_{ F_{n+1}}( u vert mu)() to the increment of the predictable part in the Doob decomposition of L() = -ln( mu()/ u()), it derives global, one-step-ahead conditions that characterize randomness notions. The work connects algorithmic randomness to classical results on merging of opinions (Blackwell–Dubins, Kalai–Lehrer) while extending the theory to Hellinger distance and KL-divergence, and it situates these results alongside Vovk’s local-to-global perspectives. The findings illuminate how effective notions of absolute continuity and information distances govern convergence of predictions and the convergence of posteriors in a computable setting, with implications for Bayesian-style inference and predictive statistics.
Abstract
We characterize Martin-Löf randomness and Schnorr randomness in terms of the merging of opinions, along the lines of the Blackwell-Dubins Theorem. After setting up a general framework for defining notions of merging randomness, we focus on finite horizon events, that is, on weak merging in the sense of Kalai-Lehrer. In contrast to Blackwell-Dubins and Kalai-Lehrer, we consider not only the total variational distance but also the Hellinger distance and the Kullback-Leibler divergence. Our main result is a characterization of Martin-Löf randomness and Schnorr randomness in terms of weak merging and the summable Kullback-Leibler divergence. The main proof idea is that the Kullback-Leibler divergence between $μ$ and $ν$, at a given stage of the learning process, is exactly the incremental growth, at that stage, of the predictable process of the Doob decomposition of the $ν$-submartingale $L(σ)=-\ln \frac{μ(σ)}{ν(σ)}$. These characterizations of algorithmic randomness notions in terms of the Kullback-Leibler divergence can be viewed as global analogues of Vovk's theorem on what transpires locally with individual Martin-Löf $μ$- and $ν$-random points and the Hellinger distance between $μ,ν$.