Derivative-free Alternating Projection Algorithms for General Nonconvex-Concave Minimax Problems
Zi Xu, Ziqi Wang, Jingjing Shen, Yuhong Dai
TL;DR
This work develops two derivative-free algorithms for nonconvex-concave minimax problems: ZO-AGP for smooth settings and ZO-BAPG for block-wise nonsmooth cases. Each method uses finite-difference gradient estimators and projection/proximal steps to achieve an $\varepsilon$-stationary point with iteration complexity $\mathcal{O}(\varepsilon^{-4})$ and per-iteration function evaluations of $\mathcal{O}(d_x+d_y)$ (or $\mathcal{O}(K d_x+d_y)$ for the block case). The authors provide rigorous complexity analyses under Lipschitz-gradient assumptions and nonincreasing regularization, plus variance bounds for the zeroth-order estimators. Numerical experiments on data poisoning and distributed SPCA validate the practical efficiency and demonstrate the advantages over existing zeroth-order baselines. Overall, the paper delivers the first comprehensive zeroth-order complexity guarantees for general nonconvex-concave minimax problems and shows their applicability to realistic machine learning tasks.
Abstract
In this paper, we study zeroth-order algorithms for nonconvex-concave minimax problems, which have attracted widely attention in machine learning, signal processing and many other fields in recent years. We propose a zeroth-order alternating randomized gradient projection (ZO-AGP) algorithm for smooth nonconvex-concave minimax problems, and its iteration complexity to obtain an $\varepsilon$-stationary point is bounded by $\mathcal{O}(\varepsilon^{-4})$, and the number of function value estimation is bounded by $\mathcal{O}(d_{x}+d_{y})$ per iteration. Moreover, we propose a zeroth-order block alternating randomized proximal gradient algorithm (ZO-BAPG) for solving block-wise nonsmooth nonconvex-concave minimax optimization problems, and the iteration complexity to obtain an $\varepsilon$-stationary point is bounded by $\mathcal{O}(\varepsilon^{-4})$ and the number of function value estimation per iteration is bounded by $\mathcal{O}(K d_{x}+d_{y})$. To the best of our knowledge, this is the first time that zeroth-order algorithms with iteration complexity gurantee are developed for solving both general smooth and block-wise nonsmooth nonconvex-concave minimax problems. Numerical results on data poisoning attack problem and distributed nonconvex sparse principal component analysis problem validate the efficiency of the proposed algorithms.