Data-Driven Performance Guarantees for Parametric Optimization Problems

TL;DR

The paper addresses the challenge of providing probabilistic guarantees for parametric optimization problems solved with iterative methods under limited computation time. It formulates convergence and fixed-iteration performance as scenario optimization problems, learning the worst-case iteration counts $n^*$ and, separately, bounds on performance metrics such as $e^k_N$ with rigorous generalization guarantees. A relaxation framework introduces trade-offs between computation time and convergence risk, using non-convex scenario optimization results to derive bounds that account for constraint relaxation via a penalty parameter $\rho$. Numerical experiments in MPC demonstrate tight probabilistic guarantees and illuminate the trade-off between iteration budget and solution quality, offering a practical approach for online optimization with data-driven assurances.

Abstract

We propose a data-driven method to establish probabilistic performance guarantees for parametric optimization problems solved via iterative algorithms. Our approach addresses two key challenges: providing convergence guarantees to characterize the worst-case number of iterations required to achieve a predefined tolerance, and upper bounding a performance metric after a fixed number of iterations. These guarantees are particularly useful for online optimization problems with limited computational time, where existing performance guarantees are often unavailable or unduly conservative. We formulate the convergence analysis problem as a scenario optimization program based on a finite set of sampled parameter instances. Leveraging tools from scenario optimization theory enables us to derive probabilistic guarantees on the number of iterations needed to meet a given tolerance level. Using recent advancements in scenario optimization, we further introduce a relaxation approach to trade the number of iterations against the risk of violating convergence criteria thresholds. Additionally, we analyze the trade-off between solution accuracy and time efficiency for fixed-iteration optimization problems by casting them into scenario optimization programs. Numerical simulations demonstrate the efficacy of our approach in providing reliable probabilistic convergence guarantees and evaluating the trade-off between solution accuracy and computational cost.

Figures (3)

• Figure 1: The black crosses indicate the optimal iteration number $n^*_N(\rho)$ and the associated bound $\epsilon(s^*_N)$ computed using Theorem 2, which together provide an upper bound on $V(n^*_N(\rho))$. The blue line shows the upper bound on $V(n_a)$, while the orange line represents the empirical violation probability. The red crosses correspond to the method described in footnote $2$.
• Figure 2: Upper bounds on the performance metric computed via \ref{['e_scenario']} using $1000$ sampled initial conditions.
• Figure 3: Trade-off between violation probability and metric bound $e^k_N(\rho)$. The black curves represent the bounds $\underline{\epsilon}(s^*_N)$ and $\bar{\epsilon}(s^*_N)$, obtained through Theorem 3, while the orange curve shows bound $\epsilon(s^*_N)$ from Theorem 2.

Theorems & Definitions (6)

• Definition II.1: Risk
• Theorem 1: campi2008exact
• Definition II.2: Consistency property, non-convex_scenario_optimization2024
• Lemma II.1
• Theorem 2
• Theorem 3