On the Functoriality of Belief Propagation Algorithms on finite Partially Ordered Sets
Grégoire Sergeant-Perthuis, Toby St Clere Smithe, Léo Boitel
TL;DR
The paper addresses inference on expressive probabilistic models by recasting graphical models as presheaves over finite posets and framing the Bethe free energy in this setting. It proves functoriality results: natural transformations between presheaves induce morphisms between corresponding message-passing algorithms, with $\Delta MP$ obeying $\Delta MP_{G,\widetilde{H}}=\phi\circ \Delta MP_{F,H}\circ \phi^{\dagger}$ and $MP$ becoming a functor under suitable inner products. These results connect topological transformations of the underlying poset and presheaf to the behavior of inference updates, enabling principled analysis of topology-driven changes to fixed points of the Bethe free energy. The findings lay groundwork for topology-aware design of inference algorithms and unify several inference formalisms (graphical models, factor graphs, presheaf-based models) under a functorial framework, with potential impacts on robust and scalable probabilistic reasoning in complex-structured domains.
Abstract
Undirected graphical models are a widely used class of probabilistic models in machine learning that capture prior knowledge or putative pairwise interactions between variables. Those interactions are encoded in a graph for pairwise interactions; however, generalizations such as factor graphs account for higher-degree interactions using hypergraphs. Inference on such models, which is performed by conditioning on some observed variables, is typically done approximately by optimizing a free energy, which is an instance of variational inference. The Belief Propagation algorithm is a dynamic programming algorithm that finds critical points of that free energy. Recent efforts have been made to unify and extend inference on graphical models and factor graphs to more expressive probabilistic models. A synthesis of these works shows that inference on graphical models, factor graphs, and their generalizations relies on the introduction of presheaves and associated invariants (homology and cohomology groups).We propose to study the impact of the transformation of the presheaves onto the associated message passing algorithms. We show that natural transformations between presheaves associated with graphical models and their generalizations, which can be understood as coherent binning of the set of values of the variables, induce morphisms between associated message-passing algorithms. It is, to our knowledge, the first result on functoriality of the Loopy Belief Propagation.