Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Bregman ADMM for Bethe variational problem

Yuehaw Khoo, Tianyun Tang, Kim-Chuan Toh

TL;DR

This work tackles the Bethe variational problem (BVP), a central non-convex optimization arising from belief propagation in graphical models. It introduces a Bregman ADMM with a KL-divergence penalty and a novel nonlinear dual update, and proves that all local minima are strictly positive, enabling a stable reformulation to a boundary-free problem. The algorithm decomposes naturally into easily solvable, parallel subproblems and is shown to converge to a KKT point of a modified BVP; the authors also extend the framework to the quantum Bethe variational problem (QBVP) with numerical validation. Empirical results on spin-glass, sensor network localization, and 2D Ising models demonstrate robust performance and practical scalability, with open-source code provided for reproducibility and future research.

Abstract

In this work, we propose a novel Bregman ADMM with nonlinear dual update to solve the Bethe variational problem (BVP), a key optimization formulation in graphical models and statistical physics. Our algorithm provides rigorous convergence guarantees, even if the objective function of BVP is non-convex and non-Lipschitz continuous on the boundary. A central result of our analysis is proving that the entries in local minima of BVP are strictly positive, effectively resolving non-smoothness issues caused by zero entries. Beyond theoretical guarantees, the algorithm possesses high level of separability and parallelizability to achieve highly efficient subproblem computation. Our Bregman ADMM can be easily extended to solve the quantum Bethe variational problem. Numerical experiments are conducted to validate the effectiveness and robustness of the proposed method. Based on this research, we have released an open-source package of the proposed method at https://github.com/TTYmath/BADMM-BVP.

A Bregman ADMM for Bethe variational problem

TL;DR

This work tackles the Bethe variational problem (BVP), a central non-convex optimization arising from belief propagation in graphical models. It introduces a Bregman ADMM with a KL-divergence penalty and a novel nonlinear dual update, and proves that all local minima are strictly positive, enabling a stable reformulation to a boundary-free problem. The algorithm decomposes naturally into easily solvable, parallel subproblems and is shown to converge to a KKT point of a modified BVP; the authors also extend the framework to the quantum Bethe variational problem (QBVP) with numerical validation. Empirical results on spin-glass, sensor network localization, and 2D Ising models demonstrate robust performance and practical scalability, with open-source code provided for reproducibility and future research.

Abstract

In this work, we propose a novel Bregman ADMM with nonlinear dual update to solve the Bethe variational problem (BVP), a key optimization formulation in graphical models and statistical physics. Our algorithm provides rigorous convergence guarantees, even if the objective function of BVP is non-convex and non-Lipschitz continuous on the boundary. A central result of our analysis is proving that the entries in local minima of BVP are strictly positive, effectively resolving non-smoothness issues caused by zero entries. Beyond theoretical guarantees, the algorithm possesses high level of separability and parallelizability to achieve highly efficient subproblem computation. Our Bregman ADMM can be easily extended to solve the quantum Bethe variational problem. Numerical experiments are conducted to validate the effectiveness and robustness of the proposed method. Based on this research, we have released an open-source package of the proposed method at https://github.com/TTYmath/BADMM-BVP.

Paper Structure

This paper contains 34 sections, 11 theorems, 171 equations, 2 figures, 1 table, 5 algorithms.

Table of Contents

  1. Introduction
  2. Graphical model and belief propagation
  3. Bethe variational problem
  4. Algorithms for Bethe variational problem
  5. Quantum Bethe variational problem
  6. Summary of our contributions
  7. Outline of the rest of the paper
  8. Notations and definitions
  9. Notations and definitions
  10. Local minima of (\ref{['BVP']})
  11. (\ref{['BVP']}) has no boundary local minima
  12. Properties of entropy regularized optimal transport
  13. Equivalent formulation of (\ref{['BVP']})
  14. Proof of Theorem \ref{['intthm']}
  15. Proof of Proposition \ref{['eqprop']}
  16. ...and 19 more sections

Key Result

Theorem 2.1

Suppose $({\bf q}^*, {\bf Q}^*)$ is a local minimizer of (BVP). Then ${\bf q}^* \geq \sigma$ and ${\bf Q}^* \geq \sigma^2 / (r^2 \exp[4M])$, where and $M$ and $d(G)$ are as defined in (defiM) and the maximum degree of $G$, respectively.

Figures (2)

  • Figure 1: Results for SNL without outlier noise. $\sigma=0.02,R=0.2,m=100.$blue+'s are predicted sensors, red✳'s are real sensors, ☆'s are anchors. RMSD$=\sqrt{\sum_{k\in [n]}\|x_k-\hat{x}_k\|^2/n},$ where $\{\hat{x}_k:\ k\in [n]\}$ are predicted locations of sensors.
  • Figure 2: Results for SNL with $5\%$ outlier noise. $\sigma=0.005,R=0.2,m=100.$blue+'s are predicted sensors, red✳'s are real sensors, ☆'s are anchors. RMSD$=\sqrt{\sum_{k\in [n]}\|x_k-\hat{x}_k\|^2/n},$ where $\{\hat{x}_k:\ k\in [n]\}$ are predicted locations of sensors.

Theorems & Definitions (25)

  • Theorem 2.1
  • Proposition 2.2
  • Lemma 2.3
  • proof
  • Lemma 2.4
  • proof
  • proof
  • proof
  • Remark 3.1
  • Theorem 3.2
  • ...and 15 more