Information geometry of Bayes computations

TL;DR

This work develops a nonparametric, dually affine information-geometric framework for Bayes computations by employing a statistical bundle over maximal exponential models, where Fisher's score acts as the velocity of curves. It introduces dual parallel transports and Weyl-type affine axioms that underpin exponential and mixture charts, enabling explicit natural-gradient calculations of the KL divergence. The paper derives derivatives for core Bayes operations on product spaces, notably marginalization and conditioning, and presents an exponential-decomposition formalism that links joint, marginal, and conditional densities through affine coordinates and KL terms. Through applications to exponential families and their marginals/conditionals, the approach yields practical differential tools for Bayesian updates and variations, with potential relevance to variational Bayes and other nonparametric Bayesian methods.

Abstract

Amari's Information Geometry is a dually affine formalism for parametric probability models. The literature proposes various nonparametric functional versions. Our approach uses classical Weyl's axioms so that the affine velocity of a one-parameter statistical model equals the classical Fisher's score. In the present note, we first offer a concise review of the notion of a statistical bundle as a set of couples of probability densities and Fisher's scores. Then, we show how the nonparametric dually affine setup deals with the basic Bayes and Kullback-Leibler divergence computations.