Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Minima and Critical Points of the Bethe Free Energy Are Invariant Under Deformation Retractions of Factor Graphs

Grégoire Sergeant-Perthuis, Léo Boitel

TL;DR

The paper addresses how the Bethe free energy landscape, specifically its minima and critical points, depends on the topology of the interaction structure in graphical models. It develops a functorial, topological framework where Bethe objectives $BT_{\mathcal{A},H}$ are studied on poset-induced marginal polytopes $\mathbb{L}(\mathcal{A})$, and GBP fixed points correspond to critical points. For posets with chains of length at most 1, deformation retracts eliminating linear or colinear points induce bijections between critical points (and minima) of the original and core Bethe free energies, via explicit bijections and updated Hamiltonians $\widetilde{H}$. The result unifies and extends collapsibility-type conclusions, showing that the Bethe landscape is invariant under homotopy-equivalent reconfigurations of the interaction graph, with potential implications for robustness of approximate inference in cyclic and complex factor graphs.

Abstract

In graphical models, factor graphs, and more generally energy-based models, the interactions between variables are encoded by a graph, a hypergraph, or, in the most general case, a partially ordered set (poset). Inference on such probabilistic models cannot be performed exactly due to cycles in the underlying structures of interaction. Instead, one resorts to approximate variational inference by optimizing the Bethe free energy. Critical points of the Bethe free energy correspond to fixed points of the associated Belief Propagation algorithm. A full characterization of these critical points for general graphs, hypergraphs, and posets with a finite number of variables is still an open problem. We show that, for hypergraphs and posets with chains of length at most 1, changing the poset of interactions of the probabilistic model to one with the same homotopy type induces a bijection between the critical points of the associated free energy. This result extends and unifies classical results that assume specific forms of collapsibility to prove uniqueness of the critical points of the Bethe free energy.

Minima and Critical Points of the Bethe Free Energy Are Invariant Under Deformation Retractions of Factor Graphs

TL;DR

The paper addresses how the Bethe free energy landscape, specifically its minima and critical points, depends on the topology of the interaction structure in graphical models. It develops a functorial, topological framework where Bethe objectives are studied on poset-induced marginal polytopes , and GBP fixed points correspond to critical points. For posets with chains of length at most 1, deformation retracts eliminating linear or colinear points induce bijections between critical points (and minima) of the original and core Bethe free energies, via explicit bijections and updated Hamiltonians . The result unifies and extends collapsibility-type conclusions, showing that the Bethe landscape is invariant under homotopy-equivalent reconfigurations of the interaction graph, with potential implications for robustness of approximate inference in cyclic and complex factor graphs.

Abstract

In graphical models, factor graphs, and more generally energy-based models, the interactions between variables are encoded by a graph, a hypergraph, or, in the most general case, a partially ordered set (poset). Inference on such probabilistic models cannot be performed exactly due to cycles in the underlying structures of interaction. Instead, one resorts to approximate variational inference by optimizing the Bethe free energy. Critical points of the Bethe free energy correspond to fixed points of the associated Belief Propagation algorithm. A full characterization of these critical points for general graphs, hypergraphs, and posets with a finite number of variables is still an open problem. We show that, for hypergraphs and posets with chains of length at most 1, changing the poset of interactions of the probabilistic model to one with the same homotopy type induces a bijection between the critical points of the associated free energy. This result extends and unifies classical results that assume specific forms of collapsibility to prove uniqueness of the critical points of the Bethe free energy.

Paper Structure

This paper contains 17 sections, 12 theorems, 87 equations.

Table of Contents

  1. Introduction
  2. The Bethe Free Energy and the General Belief Propagation algorithms
  3. Hypergraphs, Factor Graphs, and Posets
  4. Bethe Free Energy
  5. The General Belief Propagation algorithm, and existence of critical points of the Bethe Free energy for factor graphs
  6. The general setting
  7. The special case of factor graphs
  8. Background on the topology of partially ordered sets
  9. Partially ordered spaces as topological spaces
  10. Galois connection
  11. Homotopic posets and core of a poset
  12. Minima and Critical Points of the Bethe Free Energy under Deformation Retractions
  13. Under the retraction of a linear point
  14. Under retraction of colinear points
  15. The main result: minima and critical points under deformation retractions of hypergraphs
  16. ...and 2 more sections

Key Result

Proposition 1

The Belief propagation algorithm up to multiplicative constants has at least one fixed point such that all messages are strictly positive, i.e. there is a collection $\left(M^*_{a\to i}\in \mathbb{R}^{E_i}; a\in A,i\in I, \text{ s.t. } i\in a \right)$ such that,

Theorems & Definitions (34)

  • Definition 1: Factor graph kschischang2002factoryedidia2005
  • Proposition 1
  • proof
  • Corollary 1
  • proof
  • Definition 2: Galois Connection
  • Proposition 2: Theorem 4.1 WALKER1981373
  • proof
  • Definition 3: Homotopy of maps and posets
  • Definition 4: Linear and colinear points
  • ...and 24 more