Stochastic Smoothed Gradient Descent Ascent for Federated Minimax Optimization
Wei Shen, Minhui Huang, Jiawei Zhang, Cong Shen
TL;DR
This paper tackles federated minimax optimization by introducing FESS-GDA, a smoothing-based single-loop algorithm that unifies several nonconvex regimes (NC-PL, NC-SC, NC-C, NC-1PC, PL-PL) under a common framework. By incorporating a smoothing term and a proximal-like update via an auxiliary variable $z_t$, FESS-GDA achieves improved per-client sample and communication complexities across settings, and translates stationarity results for the inner problem $f$ to stationarity of the outer objective $\Phi=\max_y f(x,y)$. The authors provide extensive theoretical analysis with explicit rates, plus empirical validation on federated GAN training and fair classification, showing practical speedups and fairness benefits over strong baselines. The work highlights the effectiveness of smoothing in federated minimax optimization and offers a unified, versatile tool for privacy-preserving, communication-constrained multi-client learning.
Abstract
In recent years, federated minimax optimization has attracted growing interest due to its extensive applications in various machine learning tasks. While Smoothed Alternative Gradient Descent Ascent (Smoothed-AGDA) has proved its success in centralized nonconvex minimax optimization, how and whether smoothing technique could be helpful in federated setting remains unexplored. In this paper, we propose a new algorithm termed Federated Stochastic Smoothed Gradient Descent Ascent (FESS-GDA), which utilizes the smoothing technique for federated minimax optimization. We prove that FESS-GDA can be uniformly used to solve several classes of federated minimax problems and prove new or better analytical convergence results for these settings. We showcase the practical efficiency of FESS-GDA in practical federated learning tasks of training generative adversarial networks (GANs) and fair classification.