Dual Acceleration for Minimax Optimization: Linear Convergence Under Relaxed Assumptions
Jingwang Li, Xiao Li
TL;DR
This work studies minimax optimization with bilinear coupling and seeks linear convergence under relaxed conditions. It introduces PDPG, proving linear convergence under a weaker assumption than existing methods, and then develops iDAPG, an inexact Dual Accelerated Proximal Gradient that achieves linear convergence under even weaker conditions with explicit outer and inner iteration complexities. The analysis reveals κ-related quantities, such as κ_φ = (σ_max(B)^2 + μ_x L_y)/(μ_x μ_φ), that govern the convergence rate and shows how inner accuracy controls outer progress. Practically, iDAPG can outperform state-of-the-art methods in regimes where dual-costs dominate or when standard strong convexity assumptions do not hold, and the paper provides guidance on algorithm choice based on oracle costs.
Abstract
This paper addresses the bilinearly coupled minimax optimization problem: $\min_{x \in \mathbb{R}^{d_x}}\max_{y \in \mathbb{R}^{d_y}} \ f_1(x) + f_2(x) + y^{\top} Bx - g_1(y) - g_2(y)$, where $f_1$ and $g_1$ are smooth convex functions, $f_2$ and $g_2$ are potentially nonsmooth convex functions, and $B$ is a coupling matrix. Existing algorithms for solving this problem achieve linear convergence only under stronger conditions, which may not be met in many scenarios. We first introduce the Primal-Dual Proximal Gradient (PDPG) method and demonstrate that it converges linearly under an assumption where existing algorithms fail to achieve linear convergence. Building on insights gained from analyzing the convergence conditions of existing algorithms and PDPG, we further propose the inexact Dual Accelerated Proximal Gradient (iDAPG) method. This method achieves linear convergence under weaker conditions than those required by existing approaches. Moreover, even in cases where existing methods guarantee linear convergence, iDAPG can still provide superior theoretical performance in certain scenarios.