Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Neural Sum-of-Squares: Certifying the Nonnegativity of Polynomials with Transformers

Nico Pelleriti, Christoph Spiegel, Shiwei Liu, David Martínez-Rubio, Max Zimmer, Sebastian Pokutta

TL;DR

The paper tackles certifying polynomial nonnegativity via SOS, which requires solving SDPs whose size scales with the monomial basis. It proposes a learning-augmented pipeline that uses a Transformer to predict a compact basis within the half Newton polytope, complemented by a CoverageRepair step and a score-guided iterative expansion with verification to preserve correctness. Theoretical guarantees bound the overhead of the learning component, and extensive experiments on hundreds of polynomials up to $n=100$ variables demonstrate speedups surpassing $100\times$ and successful solving of instances where traditional baselines fail. This approach significantly improves the practical scalability of SOS programming, with potential impact across control, robotics, and nonconvex optimization domains. The work also integrates the Algorithms with Predictions framework to quantify performance under prediction errors and shows robust performance under distribution shifts.

Abstract

Certifying nonnegativity of polynomials is a well-known NP-hard problem with direct applications spanning non-convex optimization, control, robotics, and beyond. A sufficient condition for nonnegativity is the Sum of Squares (SOS) property, i.e., it can be written as a sum of squares of other polynomials. In practice, however, certifying the SOS criterion remains computationally expensive and often involves solving a Semidefinite Program (SDP), whose dimensionality grows quadratically in the size of the monomial basis of the SOS expression; hence, various methods to reduce the size of the monomial basis have been proposed. In this work, we introduce the first learning-augmented algorithm to certify the SOS criterion. To this end, we train a Transformer model that predicts an almost-minimal monomial basis for a given polynomial, thereby drastically reducing the size of the corresponding SDP. Our overall methodology comprises three key components: efficient training dataset generation of over 100 million SOS polynomials, design and training of the corresponding Transformer architecture, and a systematic fallback mechanism to ensure correct termination, which we analyze theoretically. We validate our approach on over 200 benchmark datasets, achieving speedups of over $100\times$ compared to state-of-the-art solvers and enabling the solution of instances where competing approaches fail. Our findings provide novel insights towards transforming the practical scalability of SOS programming.

Neural Sum-of-Squares: Certifying the Nonnegativity of Polynomials with Transformers

TL;DR

The paper tackles certifying polynomial nonnegativity via SOS, which requires solving SDPs whose size scales with the monomial basis. It proposes a learning-augmented pipeline that uses a Transformer to predict a compact basis within the half Newton polytope, complemented by a CoverageRepair step and a score-guided iterative expansion with verification to preserve correctness. Theoretical guarantees bound the overhead of the learning component, and extensive experiments on hundreds of polynomials up to variables demonstrate speedups surpassing and successful solving of instances where traditional baselines fail. This approach significantly improves the practical scalability of SOS programming, with potential impact across control, robotics, and nonconvex optimization domains. The work also integrates the Algorithms with Predictions framework to quantify performance under prediction errors and shows robust performance under distribution shifts.

Abstract

Certifying nonnegativity of polynomials is a well-known NP-hard problem with direct applications spanning non-convex optimization, control, robotics, and beyond. A sufficient condition for nonnegativity is the Sum of Squares (SOS) property, i.e., it can be written as a sum of squares of other polynomials. In practice, however, certifying the SOS criterion remains computationally expensive and often involves solving a Semidefinite Program (SDP), whose dimensionality grows quadratically in the size of the monomial basis of the SOS expression; hence, various methods to reduce the size of the monomial basis have been proposed. In this work, we introduce the first learning-augmented algorithm to certify the SOS criterion. To this end, we train a Transformer model that predicts an almost-minimal monomial basis for a given polynomial, thereby drastically reducing the size of the corresponding SDP. Our overall methodology comprises three key components: efficient training dataset generation of over 100 million SOS polynomials, design and training of the corresponding Transformer architecture, and a systematic fallback mechanism to ensure correct termination, which we analyze theoretically. We validate our approach on over 200 benchmark datasets, achieving speedups of over compared to state-of-the-art solvers and enabling the solution of instances where competing approaches fail. Our findings provide novel insights towards transforming the practical scalability of SOS programming.
Paper Structure (35 sections, 9 theorems, 15 equations, 11 figures, 28 tables, 4 algorithms)

This paper contains 35 sections, 9 theorems, 15 equations, 11 figures, 28 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Learning-augmented programming.
  3. Theoretical guarantees.
  4. Empirical evaluation.
  5. Related Work
  6. Sum of Squares (SOS) Optimization
  7. Machine Learning and Algorithms with Predictions for Combinatorial Optimization
  8. Notation and Preliminaries
  9. Methodology
  10. Transformer-based Basis Prediction
  11. Construction of a Valid Initial Basis
  12. Iterative Basis Expansion with Verification
  13. Experiments
  14. Experiment 1: Comparison to Baselines
  15. Experiment 2: Transformer Basis Prediction
  16. ...and 20 more sections

Key Result

Lemma 0

Let $p(\mathbf{x}) \in \mathbb{R}[x_1, \ldots, x_n]$ be a polynomial and $B \subseteq \mathcal{M}_d$ be a basis. Then, the support of $p(\mathbf{x})$ is contained in the set of all pairwise products of monomials in $B$, formally $S(p) \subseteq B \cdot B$ where $B \cdot B = \{m_i \cdot m_j : m_i, m_

Figures (11)

  • Figure 1: Overview of our approach for verification. (i) Pipeline schematic: given a polynomial, a Transformer predicts a compact basis, then the basis is adjusted to ensure necessary conditions are met, and an is solved with iterative expansion if needed. The method guarantees correctness: if a certificate exists, it will be found; otherwise, infeasibility is certified at the full basis. (ii) Speedup over baseline for sparse polynomials with $300$, $800$, and $3000$ distinct terms.
  • Figure 2: Sampled matrix structures.
  • Figure 3: Time breakdown by problem size.
  • Figure 4: Isolated Transformer performance across polynomial structures. (i) Heatmaps of failure rates over degree and structure show sparse and block-diagonal cases are easiest while dense is hardest. (ii) Histograms of false positives highlight error distributions per structure. (iii) Success probability as a function of the number of variables demonstrates monomial-embedding makes coverage consistent across variable counts.
  • Figure 5: Repair mechanism performance evaluation. (i) Global repair outcomes demonstrate the effectiveness of greedy and permutation-based repair mechanisms across polynomial structures: permutation-based approaches often generate supersets by adding many more monomials but prove less effective for dense cases, while greedy methods show more conservative monomial addition. (ii) Distribution shift evaluation results, indicating that the repair mechanisms are effective.
  • ...and 6 more figures

Theorems & Definitions (17)

  • Lemma 0
  • Remark 1
  • Lemma 1: Geometric Expansion
  • Lemma 1
  • proof
  • Lemma 1: Consistency
  • proof
  • Lemma 1: Robustness
  • proof
  • Lemma 1: Geometric Expansion
  • ...and 7 more