Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A first-order method for nonconvex-strongly-concave constrained minimax optimization

Zhaosong Lu, Sanyou Mei

TL;DR

This work tackles nonconvex-strongly-concave constrained minimax optimization by introducing a first-order augmented Lagrangian framework that leverages the strong concavity in the y-variable. A dedicated first-order minimax solver exploits a proximal-point style subproblem to achieve improved operation complexity, and the outer AL loop delivers an $O(\varepsilon)$-KKT point with complexity $O(\varepsilon^{-3.5}\log\varepsilon^{-1})$. Theoretical guarantees are complemented by numerical experiments showing faster convergence and competitive objective values versus established baselines in both unconstrained and constrained quadratic settings. The results advance the practical efficiency of solving constrained minimax problems with first-order methods under nonconvex-strongly-concave structure.

Abstract

In this paper we study a nonconvex-strongly-concave constrained minimax problem. Specifically, we propose a first-order augmented Lagrangian method for solving it, whose subproblems are nonconvex-strongly-concave unconstrained minimax problems and suitably solved by a first-order method developed in this paper that leverages the strong concavity structure. Under suitable assumptions, the proposed method achieves an \emph{operation complexity} of $O(\varepsilon^{-3.5}\log\varepsilon^{-1})$, measured in terms of its fundamental operations, for finding an $\varepsilon$-KKT solution of the constrained minimax problem, which improves the previous best-known operation complexity by a factor of $\varepsilon^{-0.5}$.

A first-order method for nonconvex-strongly-concave constrained minimax optimization

TL;DR

This work tackles nonconvex-strongly-concave constrained minimax optimization by introducing a first-order augmented Lagrangian framework that leverages the strong concavity in the y-variable. A dedicated first-order minimax solver exploits a proximal-point style subproblem to achieve improved operation complexity, and the outer AL loop delivers an -KKT point with complexity . Theoretical guarantees are complemented by numerical experiments showing faster convergence and competitive objective values versus established baselines in both unconstrained and constrained quadratic settings. The results advance the practical efficiency of solving constrained minimax problems with first-order methods under nonconvex-strongly-concave structure.

Abstract

In this paper we study a nonconvex-strongly-concave constrained minimax problem. Specifically, we propose a first-order augmented Lagrangian method for solving it, whose subproblems are nonconvex-strongly-concave unconstrained minimax problems and suitably solved by a first-order method developed in this paper that leverages the strong concavity structure. Under suitable assumptions, the proposed method achieves an \emph{operation complexity} of , measured in terms of its fundamental operations, for finding an -KKT solution of the constrained minimax problem, which improves the previous best-known operation complexity by a factor of .
Paper Structure (11 sections, 11 theorems, 94 equations, 2 tables, 3 algorithms)

This paper contains 11 sections, 11 theorems, 94 equations, 2 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Notation and terminology
  3. A first-order method for nonconvex-strongly-concave unconstrained minimax optimization
  4. A first-order augmented Lagrangian method for nonconvex-strongly-concave constrained minimax optimization
  5. Numerical results
  6. Unconstrained nonconvex-strongly-concave minimax optimization with quadratic objective
  7. Constrained nonconvex-strongly-concave minimax optimization with quadratic objective and linear constraints
  8. Proof of the main result
  9. Proof of the main results in Section \ref{['minimax']}
  10. Proof of the main results in Section \ref{['sec:main']}
  11. A modified optimal first-order method for strongly-convex-strongly-concave minimax problem

Key Result

Theorem 1

Suppose that Assumption mmax-a holds. Let ${H}^*$, $H$$D_{\rm \bf x}$, $D_{\rm \bf y}$, and ${H}_{\rm low}$ be defined in mmax-prob, mmax-D and mmax-bnd, $L_{\nabla {h}}$ be given in Assumption mmax-a, $\epsilon$, $\hat{\epsilon}_0$ and $\hat{x}^0$ be given in Algorithm mmax-alg2, and Then Algorithm mmax-alg2 terminates and outputs an $\epsilon$-primal-dual stationary point $({x_\epsilon},{y_\eps

Theorems & Definitions (29)

  • Definition 1
  • Remark 1
  • Theorem 1: Complexity of Algorithm \ref{['mmax-alg2']}
  • Remark 2
  • Remark 3
  • Remark 4
  • Remark 5
  • Definition 2
  • Theorem 2
  • Remark 6
  • ...and 19 more