Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Approximation of optimization problems with constraints through kernel Sum-Of-Squares

Pierre-Cyril Aubin-Frankowski, Alessandro Rudi

TL;DR

This work addresses optimizing convex problems with infinitely many inequality constraints in infinite-dimensional spaces by embedding the decision function into a vector-valued RKHS and converting pointwise inequalities into equalities via kernel Sum-Of-Squares (kSoS). It provides a unified convergence theory showing that SDP-based approximations are consistent with the original problem, with rates that depend on the constraint sampling fill distance $h_{\hat{X}}$ and the smoothness $s$, and faster rates when a $kSoS$ representation exists for the optimal solution. Theoretical results establish well-posedness and a nested relation between AFF and SDP feasible sets, along with a dichotomy between slow and fast convergence regimes. Numerical examples—including global minima, optimal transport, and learning constrained vector fields—demonstrate the practical benefits of kSoS constraints, particularly in enforcing invariances and reducing constraint violations. The appendix strengthens the foundation with selection theorems and lower semicontinuity results, ensuring feasible, smooth selections in set-valued constraint maps.

Abstract

Handling an infinite number of inequality constraints in infinite-dimensional spaces occurs in many fields, from global optimization to optimal transport. These problems have been tackled individually in several previous articles through kernel Sum-Of-Squares (kSoS) approximations. We propose here a unified theorem to prove convergence guarantees for these schemes. Pointwise inequalities are turned into equalities within a class of nonnegative kSoS functions. Assuming further that the functions appearing in the problem are smooth, focusing on pointwise equality constraints enables the use of scattering inequalities to mitigate the curse of dimensionality in sampling the constraints. Our approach is illustrated in learning vector fields with side information, here the invariance of a set.

Approximation of optimization problems with constraints through kernel Sum-Of-Squares

TL;DR

This work addresses optimizing convex problems with infinitely many inequality constraints in infinite-dimensional spaces by embedding the decision function into a vector-valued RKHS and converting pointwise inequalities into equalities via kernel Sum-Of-Squares (kSoS). It provides a unified convergence theory showing that SDP-based approximations are consistent with the original problem, with rates that depend on the constraint sampling fill distance and the smoothness , and faster rates when a representation exists for the optimal solution. Theoretical results establish well-posedness and a nested relation between AFF and SDP feasible sets, along with a dichotomy between slow and fast convergence regimes. Numerical examples—including global minima, optimal transport, and learning constrained vector fields—demonstrate the practical benefits of kSoS constraints, particularly in enforcing invariances and reducing constraint violations. The appendix strengthens the foundation with selection theorems and lower semicontinuity results, ensuring feasible, smooth selections in set-valued constraint maps.

Abstract

Handling an infinite number of inequality constraints in infinite-dimensional spaces occurs in many fields, from global optimization to optimal transport. These problems have been tackled individually in several previous articles through kernel Sum-Of-Squares (kSoS) approximations. We propose here a unified theorem to prove convergence guarantees for these schemes. Pointwise inequalities are turned into equalities within a class of nonnegative kSoS functions. Assuming further that the functions appearing in the problem are smooth, focusing on pointwise equality constraints enables the use of scattering inequalities to mitigate the curse of dimensionality in sampling the constraints. Our approach is illustrated in learning vector fields with side information, here the invariance of a set.
Paper Structure (9 sections, 10 theorems, 46 equations, 2 figures, 1 table)

This paper contains 9 sections, 10 theorems, 46 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Examples
  3. Finding global minima
  4. Optimal transport with kSoS
  5. Learning dynamical systems with shape constraints
  6. Theoretical results
  7. Numerical application
  8. Technical results on closedness of sets and on scattering inequalities
  9. Appendix: Selection theorem and lower semicontinuity of a set-valued map

Key Result

Theorem 1

Under generic assumptions on the smoothness $s\in\mathbb{N}$ of the functions, non-degeneracy of the constraints and a well-behaved objective function, we have that the error between the optimal value $\mathcal{L}(\bar{\mathbf f}^{AFF}_{opt-cons_intro})$ of opt-cons_intro and the optimal value $\mat where $C>0$ is an explicit constant, and $\delta_s\in \{1,+\infty\}$, in particular $\delta_s=1$ if

Figures (2)

  • Figure 1: Streamplots of the vector field in \ref{['eq:example_GenLotka']} (\ref{['VectorField_GenLotka_true']}) and of kernel solutions with different constraints (\ref{['VectorField_GenLotka_NoCons', 'VectorField_GenLotka_0Inv', 'VectorField_GenLotka_0Inv']}). Red arrows: $N=5$ noisy observations. Black points: $M=20$ samples on the constraint set $\mathscr{K}$.
  • Figure 2: Performance criteria for the reconstruction of the vector field in \ref{['eq:example_GenLotka']} as a function of the number $M$ of inequality constraints imposed

Theorems & Definitions (20)

  • Theorem : Informal main result
  • Lemma 1: kSoS interpolation
  • proof
  • Theorem 2: Well-posedness of the problems
  • proof
  • Theorem 3: Main result
  • proof
  • Lemma 4: Closed constraint sets
  • proof
  • Lemma 5: Modified rudi2020finding
  • ...and 10 more