Semi-infinite Nonconvex Constrained Min-Max Optimization
Cody Melcher, Zeinab Alizadeh, Lindsey Hiett, Afrooz Jalilzadeh, Erfan Yazdandoost Hamedani
TL;DR
The paper tackles nonconvex min-max optimization with an infinite constraint set by formulating it as a semi-infinite problem. It introduces the inexact dynamic barrier primal-dual (iDB-PD) algorithm, which uses a QP subproblem, a feasibility-indicating indicator, and inner maximization approximations to update primal and dual variables, deriving global non-asymptotic convergence guarantees under a Łojasiewicz condition with exponent $\theta\in(0,1)$. The main theoretical contributions are explicit iteration complexities for achieving $\epsilon$-stationarity, $\epsilon$-feasibility, and $\epsilon$-complementarity: $\mathcal{O}(\epsilon^{-3})$, $\mathcal{O}(\epsilon^{-6\theta})$, and $\mathcal{O}(\epsilon^{-3\theta/(1-\theta)})$ respectively, with a special PL case $\theta=1/2$ giving $\mathcal{O}(\epsilon^{-3})$ for all. Empirically, iDB-PD demonstrates robust performance on robust multitask learning with task prioritization, outperforming adaptive discretization and DRO-based baselines, and highlighting its applicability to safety-critical ML and robust optimization under infinite constraints.
Abstract
Semi-Infinite Programming (SIP) has emerged as a powerful framework for modeling problems with infinite constraints, however, its theoretical development in the context of nonconvex and large-scale optimization remains limited. In this paper, we investigate a class of nonconvex min-max optimization problems with nonconvex infinite constraints, motivated by applications such as adversarial robustness and safety-constrained learning. We propose a novel inexact dynamic barrier primal-dual algorithm and establish its convergence properties. Specifically, under the assumption that the squared infeasibility residual function satisfies the Lojasiewicz inequality with exponent $θ\in (0,1)$, we prove that the proposed method achieves $\mathcal{O}(ε^{-3})$, $\mathcal{O}(ε^{-6θ})$, and $\mathcal{O}(ε^{-3θ/(1-θ)})$ iteration complexities to achieve an $ε$-approximate stationarity, infeasibility, and complementarity slackness, respectively. Numerical experiments on robust multitask learning with task priority further illustrate the practical effectiveness of the algorithm.