Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning to Guide Local Search for MPE Inference in Probabilistic Graphical Models

Brij Malhotra, Shivvrat Arya, Tahrima Rahman, Vibhav Giridhar Gogate

TL;DR

The paper addresses the NP-hard problem of Most Probable Explanation (MPE) inference in probabilistic graphical models, particularly in regimes where the graph is fixed but evidence varies across many queries. It introduces BEACON, an amortized lookahead framework that uses an attention-based neural network to score 1-flip neighbors by their likelihood of reducing the Hamming distance to a near-optimal solution, and combines this with the exact log-likelihood gain via a convex blend controlled by a parameter $\lambda$. Training data are generated by solving training queries with anytime MPE solvers to obtain high-quality references, and labels indicate distance-reducing moves along trajectories collected by local search. Empirically, BEACON improves over standard SLS and GLS+ across 25 high-treewidth PGMs, demonstrating faster convergence and higher final log-likelihoods, with the most pronounced gains early in search and robust performance in amortized inference settings. The work highlights the practical impact of transferable, learned guidance for local search in structured probabilistic models and suggests directions for richer supervision, multi-step lookahead, and broader applicability to discrete optimization problems.

Abstract

Most Probable Explanation (MPE) inference in Probabilistic Graphical Models (PGMs) is a fundamental yet computationally challenging problem arising in domains such as diagnosis, planning, and structured prediction. In many practical settings, the graphical model remains fixed while inference must be performed repeatedly for varying evidence patterns. Stochastic Local Search (SLS) algorithms scale to large models but rely on myopic best-improvement rule that prioritizes immediate likelihood gains and often stagnate in poor local optima. Heuristics such as Guided Local Search (GLS+) partially alleviate this limitation by modifying the search landscape, but their guidance cannot be reused effectively across multiple inference queries on the same model. We propose a neural amortization framework for improving local search in this repeated-query regime. Exploiting the fixed graph structure, we train an attention-based network to score local moves by predicting their ability to reduce Hamming distance to a near-optimal solution. Our approach integrates seamlessly with existing local search procedures, using this signal to balance short-term likelihood gains with long-term promise during neighbor selection. We provide theoretical intuition linking distance-reducing move selection to improved convergence behavior, and empirically demonstrate consistent improvements over SLS and GLS+ on challenging high-treewidth benchmarks in the amortized inference setting.

Learning to Guide Local Search for MPE Inference in Probabilistic Graphical Models

TL;DR

The paper addresses the NP-hard problem of Most Probable Explanation (MPE) inference in probabilistic graphical models, particularly in regimes where the graph is fixed but evidence varies across many queries. It introduces BEACON, an amortized lookahead framework that uses an attention-based neural network to score 1-flip neighbors by their likelihood of reducing the Hamming distance to a near-optimal solution, and combines this with the exact log-likelihood gain via a convex blend controlled by a parameter . Training data are generated by solving training queries with anytime MPE solvers to obtain high-quality references, and labels indicate distance-reducing moves along trajectories collected by local search. Empirically, BEACON improves over standard SLS and GLS+ across 25 high-treewidth PGMs, demonstrating faster convergence and higher final log-likelihoods, with the most pronounced gains early in search and robust performance in amortized inference settings. The work highlights the practical impact of transferable, learned guidance for local search in structured probabilistic models and suggests directions for richer supervision, multi-step lookahead, and broader applicability to discrete optimization problems.

Abstract

Most Probable Explanation (MPE) inference in Probabilistic Graphical Models (PGMs) is a fundamental yet computationally challenging problem arising in domains such as diagnosis, planning, and structured prediction. In many practical settings, the graphical model remains fixed while inference must be performed repeatedly for varying evidence patterns. Stochastic Local Search (SLS) algorithms scale to large models but rely on myopic best-improvement rule that prioritizes immediate likelihood gains and often stagnate in poor local optima. Heuristics such as Guided Local Search (GLS+) partially alleviate this limitation by modifying the search landscape, but their guidance cannot be reused effectively across multiple inference queries on the same model. We propose a neural amortization framework for improving local search in this repeated-query regime. Exploiting the fixed graph structure, we train an attention-based network to score local moves by predicting their ability to reduce Hamming distance to a near-optimal solution. Our approach integrates seamlessly with existing local search procedures, using this signal to balance short-term likelihood gains with long-term promise during neighbor selection. We provide theoretical intuition linking distance-reducing move selection to improved convergence behavior, and empirically demonstrate consistent improvements over SLS and GLS+ on challenging high-treewidth benchmarks in the amortized inference setting.
Paper Structure (36 sections, 1 theorem, 21 equations, 27 figures, 5 tables, 1 algorithm)

This paper contains 36 sections, 1 theorem, 21 equations, 27 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Background
  4. Notation:
  5. Learning to Guide Local Search
  6. A distance-based view of progress.
  7. Learning a surrogate oracle.
  8. Data Generation for Neighbor Scoring
  9. Neural Scoring Model
  10. Inference-Time Search Guidance
  11. Experiments
  12. Local Search Strategies.
  13. BEACON Integration.
  14. Experimental Setup
  15. Amortized Inference Regime.
  16. ...and 21 more sections

Key Result

Theorem 1

Let $\{\mathbf{x}_t\}_{t \geq 0}$ be the sequence of assignments generated by 1-flip local search, with $\mathbf{x}_t$ denoting the state at step $t$. Assume the process is absorbing at the target assignment, i.e., if $\mathbf{x}_t = \mathbf{x}^\ast$ then $\mathbf{x}_{t+1} = \mathbf{x}^\ast$. Define

Figures (27)

  • Figure 1: Attention-based neural architecture for scoring 1-flip neighbors by their predicted utility in reducing the Hamming distance to a high-quality assignment.
  • Figure 2: Win percentage heatmaps comparing our methods to their respective baselines: (left) BEACON-Greedy vs. Greedy and (right) BEACON-GLS+ vs. GLS+. Columns show datasets; rows show search steps. Green indicates that the BEACON-enhanced method achieves higher log-likelihood than its baseline in majority of the cases ($>$ 50 points), red indicates lower performance, and darker shades represent larger differences.
  • Figure 3: Average log-likelihood scores across 100 MPE queries for different $\lambda$ values on the BN-30 model. The x-axis shows the step budget, and the y-axis shows the average log-likelihood with standard deviation. The left subfigure corresponds to BEACON-Greedy, and the right subfigure to BEACON-GLS+.
  • Figure 4: Average log-likelihood scores across 100 MPE queries for different $\lambda$ values on the BN-32 model. The x-axis shows the step budget, and the y-axis shows the average log-likelihood with standard deviation. The left subfigure corresponds to BEACON-Greedy, and the right subfigure to BEACON-GLS+.
  • Figure 5: Average log-likelihood scores across 100 MPE queries for different $\lambda$ values on the BN-45 model. The x-axis shows the step budget, and the y-axis shows the average log-likelihood with standard deviation. The left subfigure corresponds to BEACON-Greedy, and the right subfigure to BEACON-GLS+.
  • ...and 22 more figures

Theorems & Definitions (2)

  • Theorem 1: Convergence
  • proof