Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Theory and Practice of MAP Inference over Non-Convex Constraints

Leander Kurscheidt, Gabriele Masina, Roberto Sebastiani, Antonio Vergari

TL;DR

This work tackles MAP inference under non-convex algebraic SMT($\mathcal{LRA}$) constraints with non-log-concave densities. It introduces MpMap, an exact, scalable message-passing algorithm for tree-structured problem graphs, and PaMap, a modular framework that partitions the feasible SMT region into convex polytopes and optimizes locally, with pruning and optional SOS/summary bounds. The methods are validated on synthetic benchmarks and real-world tasks (trajectory prediction and VAE-based imputation), surpassing constraint-agnostic baselines and competing solvers in accuracy and efficiency. Overall, the paper advances practical, scalable constrained MAP inference for complex densities and non-convex constraints, with potential impact on safety-critical ML systems.

Abstract

In many safety-critical settings, probabilistic ML systems have to make predictions subject to algebraic constraints, e.g., predicting the most likely trajectory that does not cross obstacles. These real-world constraints are rarely convex, nor the densities considered are (log-)concave. This makes computing this constrained maximum a posteriori (MAP) prediction efficiently and reliably extremely challenging. In this paper, we first investigate under which conditions we can perform constrained MAP inference over continuous variables exactly and efficiently and devise a scalable message-passing algorithm for this tractable fragment. Then, we devise a general constrained MAP strategy that interleaves partitioning the domain into convex feasible regions with numerical constrained optimization. We evaluate both methods on synthetic and real-world benchmarks, showing our approaches outperform constraint-agnostic baselines, and scale to complex densities intractable for SoTA exact solvers.

The Theory and Practice of MAP Inference over Non-Convex Constraints

TL;DR

This work tackles MAP inference under non-convex algebraic SMT() constraints with non-log-concave densities. It introduces MpMap, an exact, scalable message-passing algorithm for tree-structured problem graphs, and PaMap, a modular framework that partitions the feasible SMT region into convex polytopes and optimizes locally, with pruning and optional SOS/summary bounds. The methods are validated on synthetic benchmarks and real-world tasks (trajectory prediction and VAE-based imputation), surpassing constraint-agnostic baselines and competing solvers in accuracy and efficiency. Overall, the paper advances practical, scalable constrained MAP inference for complex densities and non-convex constraints, with potential impact on safety-critical ML systems.

Abstract

In many safety-critical settings, probabilistic ML systems have to make predictions subject to algebraic constraints, e.g., predicting the most likely trajectory that does not cross obstacles. These real-world constraints are rarely convex, nor the densities considered are (log-)concave. This makes computing this constrained maximum a posteriori (MAP) prediction efficiently and reliably extremely challenging. In this paper, we first investigate under which conditions we can perform constrained MAP inference over continuous variables exactly and efficiently and devise a scalable message-passing algorithm for this tractable fragment. Then, we devise a general constrained MAP strategy that interleaves partitioning the domain into convex feasible regions with numerical constrained optimization. We evaluate both methods on synthetic and real-world benchmarks, showing our approaches outperform constraint-agnostic baselines, and scale to complex densities intractable for SoTA exact solvers.
Paper Structure (50 sections, 9 theorems, 39 equations, 23 figures, 4 tables, 15 algorithms)

This paper contains 50 sections, 9 theorems, 39 equations, 23 figures, 4 tables, 15 algorithms.

Table of Contents

  1. Intro
  2. Maximum A Posteriori Inference Under Non-Convex Algebraic Constraints
  3. Notation.
  4. SMT($\mathcal{LRA}$) Constraints.
  5. MAP under SMT constraints.
  6. The SoTA of Constrained MAP Inference
  7. Convex constraints, log-concave densities.
  8. Convex constraints and piecewise densities.
  9. General polynomial densities.
  10. Local optimizers for general densities and convex constraints.
  11. Optimization under general SMT constraints.
  12. A Scalable Message-Passing Algorithm For Constrained MAP
  13. A tree-shaped MAP($\mathcal{LRA}$) MAP($\mathcal{LRA}$) .
  14. MpMap.
  15. PaMap: scaling local convex optimizers
  16. ...and 35 more sections

Key Result

Theorem 4.5

If the global graph of MAP($\mathcal{LRA}$) MAP($\mathcal{LRA}$) has treewidth one and bounded diameter, and the density fulfills the TMC (def:tractable_mp_conditions), then MAP($\mathcal{LRA}$) MAP($\mathcal{LRA}$) can be solved tractably.

Figures (23)

  • Figure 1: Our structure-aware solver PaMap is able to correctly and efficiently perform MAP inference over non-convex constraints and non-log-concave densities while classical optimizers (Adam), even when being constrained-aware ($\textsc{PCAdam}$; \ref{['sec:experiments']}), are imprecise and slower and when exact solvers (OptiMathSAT sebastiani2020optimathsat and CDCL-OCAC jia2025complete) timeout. If the density factorizes as a tree, our MpMap solver (\ref{['sec:mp-map']}) can be exact and even faster (see \ref{['fig:mp_map_timeout_overview']}).
  • Figure 2: An example of \ref{['eq:constrained-map']} inference over non-convex constraints and non-log-concave density. Left: an unconstrained density in 2D. Center: non-convex constraints. Right: constrained density. Orange stars indicate the solutions for the MAP problem for the unconstrained and constrained densities.
  • Figure 3: Densities in $\boldsymbol{\Omega}^{\mathsf{PP}}\xspace$ can express complex and multimodal densities as shown here for the density $p_{1,2}(x_1,x_2)$ from example \ref{['ex:example_wfamily_pp']}. Despite factorizing into univariate polynomials, different pieces can recover correlations.
  • Figure 4: PaMap can decompose non-convex feasible regions into convex polytopes, as shown for the example in \ref{['fig:intro-ex']}. Note that the partitioning only depends on the constraints $\Delta$. As such, for a conditional density $p(\bm{\mathrm{y}}\,|\bm{\mathrm{x}})$, it needs to be computed only once for all datapoints $\bm{\mathrm{x}}$ as in the SDD experiments (\ref{['sec:experiments']}).
  • Figure 5: When MAP($\mathcal{LRA}$) MAP($\mathcal{LRA}$) has tree-structure, MpMap outperforms competitors such as PaMap and $\textsc{PCAdam}$ on graphs with different diameter, including PATH graphs with maximal diameter $d{=}N{-}1$, where the worst-case complexity of MpMap would scale exponentially. Details in \ref{['sec:app_details_tree_shaped_primal']}.
  • ...and 18 more figures

Theorems & Definitions (26)

  • Example 2.1: SMT formula
  • Example 2.2: Constrained MAP
  • Example 4.1: Primal graph of SMT formula
  • Definition 4.2: Tractable MAP Conditions
  • Example 4.3: Example of a density in $\boldsymbol{\Omega}^{\mathsf{PP}}\xspace$
  • Definition 4.4
  • Theorem 4.5: Tractability of MAP($\mathcal{LRA}$) MAP($\mathcal{LRA}$)
  • Example 4.6: MpMap in action
  • Example 1.1: MpMap in action
  • Example 1.2
  • ...and 16 more