Negative Stepsizes Make Gradient-Descent-Ascent Converge

TL;DR

This work interprets the proposed unconventional stepsize schedules (dubbed slingshot stepsize schedules) as a second-order finite-differencing algorithm and shows that, intriguingly, it approximately implements consensus optimization, an empirically popular algorithm for min-max problems involving deep neural networks (e.g., training GANs).

Abstract

Efficient computation of min-max problems is a central question in optimization, learning, games, and controls. Arguably the most natural algorithm is gradient-descent-ascent (GDA). However, since the 1970s, conventional wisdom has argued that GDA fails to converge even on simple problems. This failure spurred an extensive literature on modifying GDA with additional building blocks such as extragradients, optimism, momentum, anchoring, etc. In contrast, we show that GDA converges in its original form by simply using a judicious choice of stepsizes. The key innovation is the proposal of unconventional stepsize schedules (dubbed slingshot stepsize schedules) that are time-varying, asymmetric, and periodically negative. We show that all three properties are necessary for convergence, and that altogether this enables GDA to converge on the classical counterexamples (e.g., unconstrained convex-concave problems). All of our results apply to the last iterate of GDA, as is typically desired in practice. The core algorithmic intuition is that although negative stepsizes make backward progress, they de-synchronize the min and max variables (overcoming the cycling issue of GDA), and lead to a slingshot phenomenon in which the forward progress in the other iterations is overwhelmingly larger. This results in fast overall convergence. Geometrically, the slingshot dynamics leverage the non-reversibility of gradient flow: positive/negative steps cancel to first order, yielding a second-order net movement in a new direction that leads to convergence and is otherwise impossible for GDA to move in. We interpret this as a second-order finite-differencing algorithm and show that, intriguingly, it approximately implements consensus optimization, an empirically popular algorithm for min-max problems involving deep neural networks (e.g., training GANs).

Figures (5)

• Figure 1: GDA trajectories for a classical counterexample: the unconstrained bilinear problem $\min_{x \in \mathbb{R}} \max_{y \in \mathbb{R}} xy$. The unique solution is the origin. GDA fails to converge with standard stepsize schedules, e.g., it diverges if $\alpha_t=\beta_t > 0$ (red), or limit-cycles if $\alpha_t,\beta_t$ are alternately positive and zero (yellow). Our proposed stepsize schedule (blue) enables GDA to converge for the first time. Standard GDA attempts to follow the grey vector field $(-\nabla_x f, \nabla_y f) = (-y,x)$; in contrast, our proposed GDA alternately moves in the directions $(\nabla_x f, \nabla_y f) = (y,x)$ and $(-\nabla_x f, -\nabla_y f) = (-y,-x)$. This trajectory "slingshots" to make progress every two iterations.
• Figure 2: Optimal stepsize schedule for unconstrained bilinear min-max optimization, here for $16$ iterations, for a bilinear coupling with spectral bounds $1/100 \leq \sigma_i^2(\nabla_{xy}^2 f) \leq 1$, and in the stable fractal ordering discussed in \ref{['app:stability']}. Recall that $\alpha_t$ and $\beta_t$ respectively denote the stepsizes for $x$ and $y$; standard schedules take $\alpha_t,\beta_t \geq 0$ and often equal. Observe that the proposed stepsizes are time-varying, asymmetric, and alternately negative.
• Figure 3: Comparison of algorithms on a random bilinear problem. Using standard stepsize schedules, GDA either diverges (red) or cycles (yellow). Known ways of making GDA converge require changing the algorithm and do not always lead to fast convergence (purple, green). The proposed slingshot stepsize schedule requires no changes to GDA beyond stepsizes, and leads to the optimally accelerated convergence rate for bilinear problems. Performance on this random problem is similar to worst-case problems (dotted minimax lower bound); qualitatively similar behavior is observed for other problems. Further details in \ref{['app:experimental-setup']}.
• Figure 4: Illustrative numerical comparison on a $1$-smooth bilinear problem $\min_{x \in \mathbb{R}^{100}} \max_{y \in \mathbb{R}^{100}} x^\top \bm{B} y$ where $\bm{B} = \text{diag}(2^0, 2^{-1}, \dots, 2^{-99})$. All algorithms are initialized at the all-ones vector. Regardless of the stepsize choice, symmetric GDA diverges and alternating GDA cycles. Here we run the slingshot stepsize schedule for general convex-concave problems (\ref{['def:steps-nonlinear']}); using the version tailored to smooth bilinear problems (\ref{['def:quadsteps']}) leads to much faster convergence than all plotted algorithms, with $\|\nabla f(z_t)\|^2 \approx 5 \times 10^{-7}$ at the last iterate $t = 10^4$. We set $h=2/(9L)$ in \ref{['def:steps-nonlinear']} which minimizes the rate in \ref{['thm:gd-cc']}. Extragradient, optimistic GDA, and extra anchored gradient are run with their standard stepsize prescriptions. See \ref{['app:experimental-setup']} for further details.
• Figure 5: Convergence of different orderings of the proposed steps using different levels of numerical precision ("vpa" command in MATLAB). We run $512$ iterations on the random bilinear problem described in \ref{['app:experimental-setup']}.

Theorems & Definitions (28)

• Lemma 2.1: Failure of GDA with standard stepsizes
• proof
• Lemma 2.2: Universal failure of standard $\textsc{GDA}$
• Definition 3.1: Slingshot stepsize schedules for bilinear min-max optimization
• Lemma 3.2: Extremal polynomial lemma for bilinear min-max problems
• Theorem 3.3: GDA converges for bilinear min-max optimization
• proof
• Theorem 3.4: Optimality of \ref{['thm:bilinear-ub']}
• Definition 3.5: Slingshot stepsize schedule for quadratic min-max optimization
• Lemma 3.6: Extremal polynomial lemma for quadratic min-max problems