Some observations on the ambivalent role of symmetries in Bayesian inference problems
Guilhem Semerjian
TL;DR
This work analyzes how symmetries shape Bayesian inference, distinguishing beneficial versus detrimental roles based on how a symmetry group acts on the hidden signal $S$ and the observations $Y$. It formalizes the Bayesian setup, introduces invariant and equivariant concepts, and develops the idea of a quotiented distance $d_G$ to handle unobservable invariances that distort standard estimators. The authors show that when symmetries act strongly on the observations, they can simplify estimation via low-degree polynomial approximations and equivariant structures; conversely, symmetries acting mainly on the signal necessitate redefining the estimation target to operate on orbits, which complicates both analysis and computation, as illustrated by SBM, orbit recovery, and extensive-rank matrix factorization. The paper links these insights to statistical-mechanics perspectives (free energy and mutual information) and to algorithmic tools (AMP/BP), outlining open questions about spontaneous symmetry breaking in large symmetry groups and proposing directions for symmetry-aware inference strategies.
Abstract
We collect in this note some observations on the role of symmetries in Bayesian inference problems, that can be useful or detrimental depending on the way they act on the signal and on the observations. We emphasize in particular the need to gauge away unobservable invariances in the definition of a distance between a signal and its estimator, and the consequences this implies for the statistical mechanics treatment of such models, taking as a motivating example the extensive rank matrix factorization problem.