Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Two trust region type algorithms for solving nonconvex-strongly concave minimax problems

Tongliang Yao, Zi Xu

TL;DR

This work tackles nonconvex-strongly concave minimax problems by introducing two trust-region algorithms, MINIMAX-TR and MINIMAX-TRACE, that operate on the outer objective $P(x)=\max_y f(x,y)$ while using an inner gradient ascent on $y$ to estimate $\nabla P(x)$ and $\nabla^2 P(x)$. Both methods achieve the optimal $O(\epsilon^{-3/2})$-type iteration complexity to obtain a $(\epsilon,\sqrt{\epsilon})$-second-order stationary point, with MINIMAX-TRACE incorporating TRACE-style contractions/expansions for robustness and, under mild conditions, exhibiting local quadratic convergence. The analysis connects these minimax methods to the broader cubic regularization and trust-region literature, and numerical experiments show improved performance of the TRACE-based approach over existing baselines on structured minimax problems.

Abstract

In this paper, we propose a Minimax Trust Region (MINIMAX-TR) algorithm and a Minimax Trust Region Algorithm with Contractions and Expansions(MINIMAX-TRACE) algorithm for solving nonconvex-strongly concave minimax problems. Both algorithms can find an $(ε, \sqrtε)$-second order stationary point(SSP) within $\mathcal{O}(ε^{-1.5})$ iterations, which matches the best well known iteration complexity.

Two trust region type algorithms for solving nonconvex-strongly concave minimax problems

TL;DR

This work tackles nonconvex-strongly concave minimax problems by introducing two trust-region algorithms, MINIMAX-TR and MINIMAX-TRACE, that operate on the outer objective while using an inner gradient ascent on to estimate and . Both methods achieve the optimal -type iteration complexity to obtain a -second-order stationary point, with MINIMAX-TRACE incorporating TRACE-style contractions/expansions for robustness and, under mild conditions, exhibiting local quadratic convergence. The analysis connects these minimax methods to the broader cubic regularization and trust-region literature, and numerical experiments show improved performance of the TRACE-based approach over existing baselines on structured minimax problems.

Abstract

In this paper, we propose a Minimax Trust Region (MINIMAX-TR) algorithm and a Minimax Trust Region Algorithm with Contractions and Expansions(MINIMAX-TRACE) algorithm for solving nonconvex-strongly concave minimax problems. Both algorithms can find an -second order stationary point(SSP) within iterations, which matches the best well known iteration complexity.
Paper Structure (8 sections, 22 theorems, 34 equations, 1 figure, 4 algorithms)

This paper contains 8 sections, 22 theorems, 34 equations, 1 figure, 4 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Works
  4. Preliminaries
  5. A Minimax Trust Region Algorithm
  6. A Minimax Trust Region Algorithm with Contractions and Expansions
  7. Numerical Experiments
  8. Conclusions

Key Result

Lemma 2.3

[chencubic, Proposition 1] Suppose $f(x,y)$ satisfies Assumption basic assumption, then $P(x)$ has $L_P:= (\kappa+1)\ell$-Lipschitz continuous gradients. Moreover, $y^*(x)$ is well-defined and $\kappa$-Lipschitz, and $\nabla P(x) = \nabla_x f(x, y^*(x))$.

Figures (1)

  • Figure 1: Numerical results of four tested algorithms for solving \ref{['numerical problem1']} with different choices of $n$, $L$ and $\gamma$.

Theorems & Definitions (25)

  • Definition 2.2
  • Lemma 2.3
  • Definition 2.4
  • Definition 2.5
  • Lemma 2.6
  • Lemma 2.7
  • Lemma 2.8
  • Lemma 3.1
  • Theorem 3.2
  • Lemma 4.2
  • ...and 15 more