Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Riemannian Alternating Descent Ascent Algorithmic Framework for Nonconvex-Linear Minimax Problems on Riemannian Manifolds

Meng Xu, Bo Jiang, Ya-Feng Liu, Anthony Man-Cho So

TL;DR

We address nonconvex-linear minimax problems on Riemannian manifolds and introduce the Riemannian alternating descent ascent (RADA) framework, which leverages a differentiable surrogate $\Phi_k$ to capture the best response in $y$ while updating $x$ via a descent subproblem. Two practical single-loop instantiations, $\text{RADA-PGD}$ and $\text{RADA-RGD}$, are analyzed, and it is shown that both achieve $\mathcal{O}(\varepsilon^{-3})$ iterations to obtain an $\varepsilon$-RGS or an $\varepsilon$-ROS point, matching the best-known rates for this class. The paper also clarifies connections to RALM and RADMM, explaining efficiency advantages, and provides extensive numerical results on SPCA, FPCA, and SSC demonstrating superior performance. These results offer a principled, scalable approach to Riemannian minimax problems with potential extensions to stochastic and broader nonsmooth settings.

Abstract

In this paper, we consider a class of nonconvex-linear minimax problems on Riemannian manifolds, which find wide applications in machine learning and signal processing. For solving this class of problems, we develop a flexible Riemannian alternating descent ascent (RADA) algorithmic framework. Within this framework, we propose two easy-to-implement yet efficient algorithms that alternately perform one or multiple projected/Riemannian gradient descent steps and a proximal gradient ascent step at each iteration. We show that the proposed RADA algorithmic framework can find both an $\varepsilon$-Riemannian-game-stationary point and an $\varepsilon$-Riemannian-optimization-stationary point within $\mathcal{O}(\varepsilon^{-3})$ iterations, achieving the best-known iteration complexity. We also reveal intriguing similarities and differences between the algorithms developed within our proposed framework and existing algorithms, thus providing important insights into the improved efficiency of the former. Lastly, we present numerical results on sparse principal component analysis (PCA), fair PCA, and sparse spectral clustering to demonstrate the superior performance of the proposed algorithms.

A Riemannian Alternating Descent Ascent Algorithmic Framework for Nonconvex-Linear Minimax Problems on Riemannian Manifolds

TL;DR

We address nonconvex-linear minimax problems on Riemannian manifolds and introduce the Riemannian alternating descent ascent (RADA) framework, which leverages a differentiable surrogate to capture the best response in while updating via a descent subproblem. Two practical single-loop instantiations, and , are analyzed, and it is shown that both achieve iterations to obtain an -RGS or an -ROS point, matching the best-known rates for this class. The paper also clarifies connections to RALM and RADMM, explaining efficiency advantages, and provides extensive numerical results on SPCA, FPCA, and SSC demonstrating superior performance. These results offer a principled, scalable approach to Riemannian minimax problems with potential extensions to stochastic and broader nonsmooth settings.

Abstract

In this paper, we consider a class of nonconvex-linear minimax problems on Riemannian manifolds, which find wide applications in machine learning and signal processing. For solving this class of problems, we develop a flexible Riemannian alternating descent ascent (RADA) algorithmic framework. Within this framework, we propose two easy-to-implement yet efficient algorithms that alternately perform one or multiple projected/Riemannian gradient descent steps and a proximal gradient ascent step at each iteration. We show that the proposed RADA algorithmic framework can find both an -Riemannian-game-stationary point and an -Riemannian-optimization-stationary point within iterations, achieving the best-known iteration complexity. We also reveal intriguing similarities and differences between the algorithms developed within our proposed framework and existing algorithms, thus providing important insights into the improved efficiency of the former. Lastly, we present numerical results on sparse principal component analysis (PCA), fair PCA, and sparse spectral clustering to demonstrate the superior performance of the proposed algorithms.
Paper Structure (19 sections, 12 theorems, 106 equations, 3 figures, 5 tables, 4 algorithms)

This paper contains 19 sections, 12 theorems, 106 equations, 3 figures, 5 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Motivating Applications
  3. Related Works
  4. Our Contributions
  5. Organization
  6. Notation and Preliminaries
  7. Proposed Algorithmic Framework
  8. Proposed RADA Algorithmic Framework
  9. Iteration Complexity
  10. Two Efficient Algorithms for Problem \ref{['prob:p1']}
  11. Verification of the Sufficient Decrease Condition \ref{['sufficient decrease']}
  12. Connections with Existing Algorithms
  13. Connection between RADA and RALM
  14. Connection between RADA-RGD and RADMM
  15. Numerical Results
  16. ...and 4 more sections

Key Result

Theorem 1

( beck2017first) Let $g:\mathcal{E}\to(-\infty,+\infty]$ be a proper closed convex function and $\lambda>0$. Then, for any $x\in\mathcal{E}$,

Figures (3)

  • Figure 1: Stationarity gap versus runtime for two representative instances of SPCA.
  • Figure 2: Stationarity gap versus number of gradient evaluations for two representative instances of FPCA.
  • Figure 3: Stationarity gap versus number of gradient evaluations for two representative instances of SSC.

Theorems & Definitions (22)

  • Theorem 1
  • Definition 1
  • Lemma 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Lemma 2
  • Lemma 3
  • ...and 12 more