Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Categorical Belief Propagation: Sheaf-Theoretic Inference via Descent and Holonomy

Enrique ter Horst, Sridhar Mahadevan, Juan Diego Zambrano

TL;DR

The paper addresses the mismatch between semantic inference and operational BP on loopy graphs by introducing a categorical framework that isolates global cycle effects as holonomy. It develops a universal syntax via a free hypergraph category, a semiring-based semantic target, and a fibered execution layer, then shows that exact inference corresponds to effective descent across graph covers. The key contributions are the descent formulation of BP, the introduction of Holonomy-Aware Tree Compilation (HATCC) to detect and resolve cycle obstructions, and rigorous complexity and exactness results that unify tree and junction-tree exactness under topological obstructions. Practically, HATCC enables sector-wise exact inference with strong diagnostics, producing speedups over junction trees on grid and random graphs and enabling UNSAT detection in satisfiability-like problems. The approach offers a principled bridge between category theory, sheaf theory, and graphical model inference with potential applications in discrete synchronization, constraint satisfaction, and multi-sensor fusion where cycle-induced inconsistencies arise.

Abstract

We develop a categorical foundation for belief propagation on factor graphs. We construct the free hypergraph category \(\Syn_Σ\) on a typed signature and prove its universal property, yielding compositional semantics via a unique functor to the matrix category \(\cat{Mat}_R\). Message-passing is formulated using a Grothendieck fibration \(\int\Msg \to \cat{FG}_Σ\) over polarized factor graphs, with schedule-indexed endomorphisms defining BP updates. We characterize exact inference as effective descent: local beliefs form a descent datum when compatibility conditions hold on overlaps. This framework unifies tree exactness, junction tree algorithms, and loopy BP failures under sheaf-theoretic obstructions. We introduce HATCC (Holonomy-Aware Tree Compilation), an algorithm that detects descent obstructions via holonomy computation on the factor nerve, compiles non-trivial holonomy into mode variables, and reduces to tree BP on an augmented graph. Complexity is \(O(n^2 d_{\max} + c \cdot k_{\max} \cdot δ_{\max}^3 + n \cdot δ_{\max}^2)\) for \(n\) factors and \(c\) fundamental cycles. Experimental results demonstrate exact inference with significant speedup over junction trees on grid MRFs and random graphs, along with UNSAT detection on satisfiability instances.

Categorical Belief Propagation: Sheaf-Theoretic Inference via Descent and Holonomy

TL;DR

The paper addresses the mismatch between semantic inference and operational BP on loopy graphs by introducing a categorical framework that isolates global cycle effects as holonomy. It develops a universal syntax via a free hypergraph category, a semiring-based semantic target, and a fibered execution layer, then shows that exact inference corresponds to effective descent across graph covers. The key contributions are the descent formulation of BP, the introduction of Holonomy-Aware Tree Compilation (HATCC) to detect and resolve cycle obstructions, and rigorous complexity and exactness results that unify tree and junction-tree exactness under topological obstructions. Practically, HATCC enables sector-wise exact inference with strong diagnostics, producing speedups over junction trees on grid and random graphs and enabling UNSAT detection in satisfiability-like problems. The approach offers a principled bridge between category theory, sheaf theory, and graphical model inference with potential applications in discrete synchronization, constraint satisfaction, and multi-sensor fusion where cycle-induced inconsistencies arise.

Abstract

We develop a categorical foundation for belief propagation on factor graphs. We construct the free hypergraph category on a typed signature and prove its universal property, yielding compositional semantics via a unique functor to the matrix category . Message-passing is formulated using a Grothendieck fibration over polarized factor graphs, with schedule-indexed endomorphisms defining BP updates. We characterize exact inference as effective descent: local beliefs form a descent datum when compatibility conditions hold on overlaps. This framework unifies tree exactness, junction tree algorithms, and loopy BP failures under sheaf-theoretic obstructions. We introduce HATCC (Holonomy-Aware Tree Compilation), an algorithm that detects descent obstructions via holonomy computation on the factor nerve, compiles non-trivial holonomy into mode variables, and reduces to tree BP on an augmented graph. Complexity is \(O(n^2 d_{\max} + c \cdot k_{\max} \cdot δ_{\max}^3 + n \cdot δ_{\max}^2)\) for factors and fundamental cycles. Experimental results demonstrate exact inference with significant speedup over junction trees on grid MRFs and random graphs, along with UNSAT detection on satisfiability instances.
Paper Structure (52 sections, 21 theorems, 136 equations, 15 figures, 6 algorithms)

This paper contains 52 sections, 21 theorems, 136 equations, 15 figures, 6 algorithms.

Table of Contents

  1. Introduction
  2. Background and conventions (condensed)
  3. Notation.
  4. What we keep in the main text.
  5. Belief Propagation as an Endomorphism on Message Fibers
  6. Factor potentials and the model
  7. Message indexing and neighborhoods
  8. The belief propagation update rules
  9. The belief propagation operator
  10. Visual representation of belief propagation
  11. Fixed points and beliefs
  12. Gauge Equivariance: How BP Interacts with Message Rescaling
  13. The gauge propagation map
  14. Main theorem: BP is semi-equivariant
  15. Visual representation of gauge symmetry
  16. ...and 37 more sections

Key Result

Proposition 3.12

The operators $T_G$, $U_h$, and $T_G^{(s)}$ are well-defined functions $\mathbf{Msg}(G) \to \mathbf{Msg}(G)$. However, these operators are not$R$-linear (not semimodule homomorphisms). Instead, they are multilinear/polynomial.

Figures (15)

  • Figure 1: Message space $\mathbf{Msg}(G)$ for a chain MRF. Each directed half-edge carries a message (function from state space to $R$). Blue arrows: variable-to-factor messages. Red arrows: factor-to-variable messages. Belief propagation iteratively updates all messages.
  • Figure 2: Variable-to-factor BP update. Variable $v$ aggregates incoming messages (red arrows) from all neighboring factors except $f$, multiplying them pointwise. This product becomes the outgoing message $v \to f$ (blue arrow).
  • Figure 3: Factor-to-variable BP update. Factor $f$ collects incoming messages (blue arrows) from all neighbors except $v$, multiplies by its potential $\phi_f$, and marginalizes (sums) over all variables except $v$. Result is the outgoing message $f \to v$ (red arrow).
  • Figure 4: Belief propagation as endomorphism $T_G : \mathbf{Msg}(G) \to \mathbf{Msg}(G)$. Starting from initial messages $m^{(0)}$, iterate $m^{(t+1)} = T_G(m^{(t)})$. Fixed points $m^* = T_G(m^*)$ encode (approximate) marginal beliefs. On trees, BP converges to exact marginals.
  • Figure 5: Gauge group $K_G$ acts on message space $\mathbf{Msg}(G)$ by pointwise rescaling. Each orbit (dashed ellipse) represents messages equivalent up to gauge. Projective message space $\mathbb{P}\mathbf{Msg}(G)$ is the quotient by this action, identifying physically equivalent message configurations.
  • ...and 10 more figures

Theorems & Definitions (88)

  • Definition 3.1: Factor potential assignment
  • Example 3.2: Pairwise MRF potentials
  • Definition 3.3: Message indexing
  • Definition 3.4: Neighborhoods
  • Definition 3.5: Variable-to-factor update
  • Definition 3.6: Factor-to-variable update
  • Example 3.7: Factor update for pairwise MRF
  • Definition 3.8: Unified local update
  • Definition 3.9: Parallel (synchronous) belief propagation
  • Definition 3.10: Single-edge (asynchronous) update
  • ...and 78 more