An accelerated first-order regularized momentum descent ascent algorithm for stochastic nonconvex-concave minimax problems

TL;DR

An accelerated first-order regularized momentum descent ascent algorithm (FORMDA) for solving stochastic nonconvex-concave minimax problems and achieves the best-known complexity bound for single-loop algorithms under the stationarity of the objective function.

Abstract

Stochastic nonconvex minimax problems have attracted wide attention in machine learning, signal processing and many other fields in recent years. In this paper, we propose an accelerated first-order regularized momentum descent ascent algorithm (FORMDA) for solving stochastic nonconvex-concave minimax problems. The iteration complexity of the algorithm is proved to be $\tilde{\mathcal{O}}(\varepsilon ^{-6.5})$ to obtain an $\varepsilon$-stationary point, which achieves the best-known complexity bound for single-loop algorithms to solve the stochastic nonconvex-concave minimax problems under the stationarity of the objective function.

Figures (2)

• Figure 1: Performance of the PG-SMD algorithm, the SGDA algorithm and the FORMDA algorithm for WGAN problem.
• Figure 2: Performance of three algorithms for solving robust multi-task learning problem.

Theorems & Definitions (25)

• Definition 2.1
• Lemma 2.1
• proof
• Lemma 2.2
• proof
• Lemma 2.3
• proof
• Lemma 2.4
• proof
• Lemma 2.5