Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhanced Adaptive Gradient Algorithms for Nonconvex-PL Minimax Optimization

Feihu Huang, Chunyu Xuan, Xinrui Wang, Siqi Zhang, Songcan Chen

TL;DR

This work tackles stochastic minimax optimization where the primal is nonconvex and the dual satisfies a Polyak-Lojasiewicz condition, with nonsmooth regularization on the primal. It introduces two enhanced momentum-based gradient descent ascent algorithms, MSGDA and AdaMSGDA, with the latter supporting flexible adaptive learning rates via matrices $A_t$ and $B_t$ and leveraging STORM-style variance reduction. They establish the best-known $ ilde{O}(\epsilon^{-3})$ sample complexity for finding an $\epsilon$-stationary point without large batches, under mild NC-PL assumptions, and provide rigorous convergence analyses via Lyapunov-type potential functions. Numerical results on Polyak-Lojasiewicz games and Wasserstein GANs confirm accelerated convergence and superior performance relative to existing gradient-based minimax methods. This work offers scalable, adaptive optimization tools for nonconvex-nonconcave minimax problems prevalent in robust learning, generative modeling, and RL.

Abstract

Minimax optimization recently is widely applied in many machine learning tasks such as generative adversarial networks, robust learning and reinforcement learning. In the paper, we study a class of nonconvex-nonconcave minimax optimization with nonsmooth regularization, where the objective function is possibly nonconvex on primal variable $x$, and it is nonconcave and satisfies the Polyak-Lojasiewicz (PL) condition on dual variable $y$. Moreover, we propose a class of enhanced momentum-based gradient descent ascent methods (i.e., MSGDA and AdaMSGDA) to solve these stochastic nonconvex-PL minimax problems. In particular, our AdaMSGDA algorithm can use various adaptive learning rates in updating the variables $x$ and $y$ without relying on any specifical types. Theoretically, we prove that our methods have the best known sample complexity of $\tilde{O}(ε^{-3})$ only requiring one sample at each loop in finding an $ε$-stationary solution. Some numerical experiments on PL-game and Wasserstein-GAN demonstrate the efficiency of our proposed methods.

Enhanced Adaptive Gradient Algorithms for Nonconvex-PL Minimax Optimization

TL;DR

This work tackles stochastic minimax optimization where the primal is nonconvex and the dual satisfies a Polyak-Lojasiewicz condition, with nonsmooth regularization on the primal. It introduces two enhanced momentum-based gradient descent ascent algorithms, MSGDA and AdaMSGDA, with the latter supporting flexible adaptive learning rates via matrices and and leveraging STORM-style variance reduction. They establish the best-known sample complexity for finding an -stationary point without large batches, under mild NC-PL assumptions, and provide rigorous convergence analyses via Lyapunov-type potential functions. Numerical results on Polyak-Lojasiewicz games and Wasserstein GANs confirm accelerated convergence and superior performance relative to existing gradient-based minimax methods. This work offers scalable, adaptive optimization tools for nonconvex-nonconcave minimax problems prevalent in robust learning, generative modeling, and RL.

Abstract

Minimax optimization recently is widely applied in many machine learning tasks such as generative adversarial networks, robust learning and reinforcement learning. In the paper, we study a class of nonconvex-nonconcave minimax optimization with nonsmooth regularization, where the objective function is possibly nonconvex on primal variable , and it is nonconcave and satisfies the Polyak-Lojasiewicz (PL) condition on dual variable . Moreover, we propose a class of enhanced momentum-based gradient descent ascent methods (i.e., MSGDA and AdaMSGDA) to solve these stochastic nonconvex-PL minimax problems. In particular, our AdaMSGDA algorithm can use various adaptive learning rates in updating the variables and without relying on any specifical types. Theoretically, we prove that our methods have the best known sample complexity of only requiring one sample at each loop in finding an -stationary solution. Some numerical experiments on PL-game and Wasserstein-GAN demonstrate the efficiency of our proposed methods.
Paper Structure (14 sections, 14 theorems, 111 equations, 7 figures, 1 table, 2 algorithms)

This paper contains 14 sections, 14 theorems, 111 equations, 7 figures, 1 table, 2 algorithms.

Table of Contents

  1. Introduction
  2. Methods
  3. MSGDA Algorithm for NC-PL Minimax Optimization
  4. AdaMSGDA Algorithm for NC-PL Minimax Optimization
  5. Convergence Analysis
  6. Convergence Analysis of MSGDA Algorithm
  7. Convergence Analysis of AdaMSGDA Algorithm
  8. Numerical Experiments
  9. Polyak-Lojasiewicz Game
  10. Wasserstein GAN
  11. Conclusion
  12. Appendix
  13. Convergence Analysis of MSGDA Algorithm
  14. Convergence Analysis of AdaMSGDA Algorithm

Key Result

Lemma 1

(Lemma A.5 of nouiehed2019solving) Let $F(x)= f(x,y^*(x))=\max_y f(x,y)$ with $y^*(x) \in \arg\max_y f(x,y)$. Under the above Assumptions ass:1-ass:2, $\nabla F(x)=\nabla_x f(x,y^*(x))$ and $F(x)$ is $L$-smooth, i.e., where $L=L_f(1+\frac{\kappa}{2})$ with $\kappa=\frac{L_f}{\mu}$.

Figures (7)

  • Figure 1: Distance to saddle point without $L_1$ regularization:$\mu=10^{-5}$ (left), $\mu=10^{-9}$ (right).
  • Figure 2: Norm of gradient without $L_1$ regularization:$\mu=10^{-5}$ (left), $\mu=10^{-9}$ (right).
  • Figure 3: Distance to saddle point with $L_1$ regularization:$\mu=10^{-5}$ (left), $\mu=10^{-9}$ (right).
  • Figure 4: Norm of gradient with $L_1$ regularization :$\mu=10^{-5}$ (left), $\mu=10^{-9}$ (right).
  • Figure 5: Training result of a Wasserstein GAN with linear generator approximating a one-dimensional Gaussian distribution
  • ...and 2 more figures

Theorems & Definitions (23)

  • Lemma 1
  • Lemma 2
  • Theorem 1
  • Remark 1
  • Lemma 3
  • Theorem 2
  • Remark 2
  • Lemma 4
  • Lemma 5
  • Lemma 6
  • ...and 13 more