Belief propagation for general graphical models with loops

TL;DR

This work presents a unification framework, TE and MITE, that extends belief propagation to arbitrary graphical models with loops and unifies existing loopy-BP approaches (BlockBP, TNMP, N-MITE) under a single formalism. It provides explicit tree-equivalent and interleaved-tree inference equations applicable to networks and tensor networks, enabling accurate computation of marginals, energy, entropy, and partition functions with modest overhead. Numerical experiments on synthetic tensor-networks and real topologies show accuracy improvements of several orders of magnitude for marginals, validating the practical benefits for decoding tasks in quantum error correction and beyond. The framework clarifies when scheduling helps BP, connects scheduling to inference structure, and enables applying loopy-BP methods to quantum and classical inference problems with loopy topologies, offering a principled trade-off between accuracy and computation.

Abstract

There is an increasing interest in scaling tensor network methods through belief propagation (BP), as well as increasing the accuracy of BP through tensor network methods. We develop a unification framework that takes an arbitrary graphical model with loops and provides message passing update rules and inference equations. We show that recent state-of-the-art methods regarding tensors and BP, like block belief propagation and tensor network message passing, are special instances of our framework. From a practical perspective, we discuss how our framework can be useful to understand the benefits of scheduling in BP, and show how it can be used for decoding purposes in quantum error correction. We simulate the computation of marginals, internal energy, Shannon entropy and the partition function on synthetic topologies (Kagome lattice and lattices resembling quantum error-correcting codes) and a real world topology of a power grid. The results show orders of magnitude accuracy increases for modest computational overheads. For the marginals, for example, we show that our framework can achieve an accuracy improvement of more than six orders of magnitude over tensor network BP.

Figures (10)

• Figure 1: Graphical models: (a) general graphical model -- the degree of the nodes is arbitrary; (b) Network -- the degree of the factor nodes is always one or two; (c) Tensor network -- the degree of the variable nodes is always one or two. The squares are factors and the circles are variables. An edge is connecting a factor-variable pair whenever the factor depends on the variable. In the case of tensor networks, dangling edges (red) are associated with variable nodes of degree one.
• Figure 2: Factor graphs: (a) the factor graph of a network; (b) simplified graph of the network; (c) factor graph of a tensor network; (d) Simplified graph of the tensor network. Note that we have not merged double edges in the simplified network graphs. We do so since merging them does not affect the definition of the neighborhoods, and hence it does not modify the message passing method in any meaningful way.
• Figure 3: Neighborhoods definition. The neighborhood around variable $i$$N_i^{(\ell_0)}$ ($N_i^{(1)}$ in blue) includes node $i$ plus the edges that have it as endpoint the nearest neighbors of $i$ in $\mathcal{V}$, together with the edges and nodes that belong to a path of length $\ell_0$ or less connecting two nearest neighbors of $i$. The neighborhood difference from variable $i$ to $j$, $N_{i \setminus j}^{(\ell_0)}$ ($N_{k \setminus j}^{(3)}$ in red), consists of $i$ together with all the edges that belong to $N_i^{\ell_0}$ and are not included in $N_j^{\ell_0}$ plus the nodes at their ends.
• Figure 4: An improvement on the BlockBP: a) Original graph; b) an instance of BlockBP; c) an instance of an N-MITE-like algorithm applied on the original graph.
• Figure 5: Synthetic tensor networks used to benchmark our generalization: (a) the square-$d_0$ lattice; (b) the square-$d_1$ lattice; (c) the square-$d_2$ lattice; (d) The Kagome lattice. We show the 4x4 instance for each of them.

Theorems & Definitions (5)

• Remark
• Remark
• Remark
• Remark : Monotonicity in $\ell_0$
• Remark : Loop structure within lattices