Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile

TL;DR

The paper analyzes convergence of first-order methods for non-monotone saddle-point problems by introducing the coherence property, linking SPs to variational inequalities. It shows vanilla mirror descent can diverge under null coherence, while optimistic mirror descent with an extra-gradient step achieves convergence across coherent problems, including stochastic settings. The authors develop a comprehensive convergence theory for MD and OMD and validate predictions with GAN experiments on Gaussian mixtures, CelebA, and CIFAR-10, demonstrating stability improvements. The work provides principled, implementable guidance for training GANs and other non-monotone SPs, suggesting optimism as a practical stabilizer.

Abstract

Owing to their connection with generative adversarial networks (GANs), saddle-point problems have recently attracted considerable interest in machine learning and beyond. By necessity, most theoretical guarantees revolve around convex-concave (or even linear) problems; however, making theoretical inroads towards efficient GAN training depends crucially on moving beyond this classic framework. To make piecemeal progress along these lines, we analyze the behavior of mirror descent (MD) in a class of non-monotone problems whose solutions coincide with those of a naturally associated variational inequality - a property which we call coherence. We first show that ordinary, "vanilla" MD converges under a strict version of this condition, but not otherwise; in particular, it may fail to converge even in bilinear models with a unique solution. We then show that this deficiency is mitigated by optimism: by taking an "extra-gradient" step, optimistic mirror descent (OMD) converges in all coherent problems. Our analysis generalizes and extends the results of Daskalakis et al. (2018) for optimistic gradient descent (OGD) in bilinear problems, and makes concrete headway for establishing convergence beyond convex-concave games. We also provide stochastic analogues of these results, and we validate our analysis by numerical experiments in a wide array of GAN models (including Gaussian mixture models, as well as the CelebA and CIFAR-10 datasets).

Figures (6)

• Figure 1: Mirror descent (MDMD) in the non-monotone saddle-point problem $f(x_{1},x_{2}) = (x_{1} - 1/2) (x_{2} - 1/2) + \frac{1}{3} \exp(-(x_{1}-1/4)^{2} - (x_{2}-3/4)^{2})$. Left: vanilla MD spirals outwards; right: optimistic MD converges.
• Figure 2: Different algorithmic benchmarks (RMSprop and Adam): adding an extra-gradient step allows the training method to accurately learn the target data distribution and eliminates cycling and oscillatory instabilities.
• Figure 3: Left: Inception score (left) and Fréchet distance (right) on CIFAR-10 when training with Adam (with and without an extra-gradient step). Results are averaged over $8$ sample runs with different random seeds.
• Figure 4: Samples generated by Adam with an extra-gradient step on CelebA (left) and CIFAR-10 (right).
• Figure 5: Trajectories of vanilla and optimistic MD in a zero-sum game of Matching Pennies (left and right respectively). Colors represent the contours of the objective, $f(x_{1},x_{2}) = (x_{1} - 1/2) (x_{2} - 1/2)$.

Theorems & Definitions (37)

• Definition 2.1
• Example 3.1: Euclidean projections
• Example 3.2: Entropic regularization
• Theorem 3.1
• Corollary 3.2
• Corollary 3.3
• Theorem 4.1
• Corollary 4.2
• Theorem 4.3
• Proposition A.1