Belief Propagation Converges to Gaussian Distributions in Sparsely-Connected Factor Graphs

TL;DR

The paper addresses when Gaussian belief propagation (GBP) is a valid approximation in highly non-Gaussian, sparsely-connected graphs typical of spatial AI. It develops a CLT-based theoretical framework showing that repeated convolution along deep, sparsely-connected paths drives BP beliefs toward Gaussianity, under four key assumptions, with an anchoring effect from strong priors. The authors prove Gaussian convergence for chain, tree, and loopy graphs and validate the theory with synthetic experiments and a challenging stereo depth estimation task, where GBP yields Gaussian-like beliefs in most regions and closely tracks BP in performance. This work justifies using GBP in a broader class of non-Gaussian problems, enabling efficient, distributed inference in realistic spatial AI systems and suggesting pathways for hybrid Gaussian/non-Gaussian inference strategies.

Abstract

Belief Propagation (BP) is a powerful algorithm for distributed inference in probabilistic graphical models, however it quickly becomes infeasible for practical compute and memory budgets. Many efficient, non-parametric forms of BP have been developed, but the most popular is Gaussian Belief Propagation (GBP), a variant that assumes all distributions are locally Gaussian. GBP is widely used due to its efficiency and empirically strong performance in applications like computer vision or sensor networks - even when modelling non-Gaussian problems. In this paper, we seek to provide a theoretical guarantee for when Gaussian approximations are valid in highly non-Gaussian, sparsely-connected factor graphs performing BP (common in spatial AI). We leverage the Central Limit Theorem (CLT) to prove mathematically that variables' beliefs under BP converge to a Gaussian distribution in complex, loopy factor graphs obeying our 4 key assumptions. We then confirm experimentally that variable beliefs become increasingly Gaussian after just a few BP iterations in a stereo depth estimation task.

Figures (8)

• Figure 1: Along a chain of variables connected with random factors, beliefs become progressively Gaussian. Variable $x_1$ has the only unary factor $f_1$ and inherits its distribution, but the variable beliefs of $x_2, x_3$ and $x_4$ are increasingly Gaussian, demonstrating the CLT's influence.
• Figure 2: BP drives variable beliefs to a Gaussian distribution for a range of graph topologies.a) shows this for a singly connected chain factor graph, connected by pairwise factors with one prior factor at the first variable. b) shows this for a singly connected tree graph where variables can be connected to more than two pairwise factors, with prior factors connected to the leaf nodes. In this case BP passes information from the leaf to root nodes. c) shows the same effect in a loopy graph connected by pairwise factors with a single prior factor attached to the first variable. BP is run iteratively here. In each of these different topologies, we see the variable beliefs converge to a Gaussian within a short distance of the closest prior factor.
• Figure 3: Unwrapping a loopy factor graph into an equivalent tree graph.(a) Loopy graph with three variables. (b) Depth-1 tree rooted at $x_1$: after one BP iteration, $b_{x_1}$ matches (a). (c) Extending the tree with identical local neighbourhoods yields matching beliefs for $x_1,x_2,x_3$ after one iteration; after two iterations, $x_1$ in (c) remains consistent with (a).
• Figure 4: Empirical Validation of the Convolutional CLT Mechanism. (a) Convergence Rate: In sparse graphs, Gaussianity is driven by topological depth (convolution). All graphs converge to a Gaussian ($D_{KL} < 0.02$) within 3 hops, and Loopy grids (green) converge to a Gaussian faster than trees (orange) or chains (blue) due to multiple feedback paths. (a) Node Degree: In a Star graph without convolutional depth, increasing the node degree increases the non-Gaussianity of the central belief. This confirms that unlike in dense "large system" limits comms_gaussian_interference, Gaussianity in sparse spatial AI graphs arises from path depth, not nodal averaging. (c) Prior Factor Uncertainty: Validating Lemma \ref{['lem:prior_strength']}, variable beliefs remain non-Gaussian ($D_{KL} > 0.02$) only when anchored by high-confidence priors (Normalized Variance $< 0.5$). As prior uncertainty increases, the convolution mechanism dominates, driving $D_{KL}$ to zero. (d) GBP Accuracy: Despite the non-Gaussian nature of the stereo depth problem (Cones scene), GBP serves as an accurate surrogate optimiser for non-parametric BP, converging to the same final MSE with negligible approximation error.
• Figure 5: Stereo Depth Estimation: BP tends to Gaussian beliefs under weak priors. In the Cones scene middlebury, BP yields Gaussian-like variable beliefs ($D_{KL} < 0.02$, green) for pixels with high-variance priors, while confident-prior pixels such as edges remain non-Gaussian ($D_{KL} > 0.02$, red).

Theorems & Definitions (5)

• Theorem 4.1: Chain Convergence
• Lemma 4.2
• Lemma 4.3
• Theorem 4.4
• Theorem 4.5