Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Projection-Free Algorithms for Minimax Problems

Khanh-Hung Giang-Tran, Soroosh Shafiee, Nam Ho-Nguyen

Abstract

This paper addresses constrained smooth saddle-point problems in settings where projection onto the feasible sets is computationally expensive. We bridge the gap between projection-based and projection-free optimization by introducing a unified dual dynamic smoothing framework that enables the design of efficient single-loop algorithms. Within this framework, we establish convergence results for nonconvex-concave and nonconvex-strongly concave settings. Furthermore, we show that this framework is naturally applicable to convex-concave problems. We propose and analyze three algorithmic variants based on the application of a linear minimization oracle over the minimization variable, the maximization variable, or both. Notably, our analysis yields anytime convergence guarantees without requiring a pre-specified iteration horizon. These results significantly narrow the performance gap between projection-free and projection-based methods for minimax optimization.

Projection-Free Algorithms for Minimax Problems

Abstract

This paper addresses constrained smooth saddle-point problems in settings where projection onto the feasible sets is computationally expensive. We bridge the gap between projection-based and projection-free optimization by introducing a unified dual dynamic smoothing framework that enables the design of efficient single-loop algorithms. Within this framework, we establish convergence results for nonconvex-concave and nonconvex-strongly concave settings. Furthermore, we show that this framework is naturally applicable to convex-concave problems. We propose and analyze three algorithmic variants based on the application of a linear minimization oracle over the minimization variable, the maximization variable, or both. Notably, our analysis yields anytime convergence guarantees without requiring a pre-specified iteration horizon. These results significantly narrow the performance gap between projection-free and projection-based methods for minimax optimization.

Paper Structure

This paper contains 43 sections, 52 theorems, 338 equations, 1 figure, 4 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. LMO-Based Methods
  4. Convex-Concave Payoff Function.
  5. Nonconvex-Concave Payoff Function.
  6. PO-Based Methods
  7. Convex-Concave Payoff Function.
  8. Nonconvex-Concave Payoff Function
  9. Other Settings
  10. Bilinear Payoff Functions.
  11. Lagrangian-Based Payoff Functions.
  12. Contributions
  13. Assumptions and Convergence Criteria
  14. A Fully LMO Algorithm
  15. A Hybrid LMO-PO Algorithm
  16. ...and 28 more sections

Key Result

Lemma 1.3

If assum holds then for any $(x,y) \in {\cal X} \times {\cal Y}$ and $\sigma > 0$, it holds that

Figures (1)

  • Figure 1: Stationarity measure over time across datasets: $\texttt{DL}$, $\texttt{RCV1}$ and $\texttt{NEWS20}$ (left to right).

Theorems & Definitions (99)

  • Lemma 1.3
  • Lemma 1.5
  • Theorem 2.1
  • Theorem 2.2
  • Theorem 3.1
  • Theorem 3.2
  • Theorem 4.1
  • Theorem 4.2
  • Lemma 1.1
  • proof
  • ...and 89 more