Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Strengthening the finite characterizations of smooth min-max games

Valery Krivchenko, Alexander Gasnikov, Dmitry Kovalev

Abstract

In this paper, we address the problem of interpolation of smooth convex-concave functions. Interpolation is a key step for computer-assisted estimation of worst-case performance via PEP-like techniques, and smooth convex-concave functions are frequently used to model min-max problems arising in machine learning. We address the challenges associated with deriving conditions that are necessary and sufficient for the interpolation of smooth min-max games and show how existing approaches can be adapted to this setting. As part of this effort, we study the smoothing properties of Moreau-Yosida-like approximations of convex-concave functions. Next, we derive interpolation conditions for several key special cases of smooth min-max games. Finally, we obtain improved (i.e., tighter) characterizations for smooth strongly monotone convex-concave functions. We analyze the linear convergence of Alt-GDA using a PEP-like technique with novel constraints and (numerically) obtain a new estimate of its complexity. We are confident that the results of this paper provide meaningful progress toward establishing optimal worst-case guarantees for algorithms in the setting of smooth min-max games.

Strengthening the finite characterizations of smooth min-max games

Abstract

In this paper, we address the problem of interpolation of smooth convex-concave functions. Interpolation is a key step for computer-assisted estimation of worst-case performance via PEP-like techniques, and smooth convex-concave functions are frequently used to model min-max problems arising in machine learning. We address the challenges associated with deriving conditions that are necessary and sufficient for the interpolation of smooth min-max games and show how existing approaches can be adapted to this setting. As part of this effort, we study the smoothing properties of Moreau-Yosida-like approximations of convex-concave functions. Next, we derive interpolation conditions for several key special cases of smooth min-max games. Finally, we obtain improved (i.e., tighter) characterizations for smooth strongly monotone convex-concave functions. We analyze the linear convergence of Alt-GDA using a PEP-like technique with novel constraints and (numerically) obtain a new estimate of its complexity. We are confident that the results of this paper provide meaningful progress toward establishing optimal worst-case guarantees for algorithms in the setting of smooth min-max games.
Paper Structure (18 sections, 25 theorems, 98 equations, 3 figures, 2 algorithms)

This paper contains 18 sections, 25 theorems, 98 equations, 3 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Motivation and prior work on interpolation
  3. Preliminaries
  4. Convex--concave interpolation and properties
  5. Review of convex--concave interpolation
  6. Basic convex--concave properties and strongly monotone case
  7. Smooth convex--concave interpolation: complications
  8. Connection between smooth and co-coercive classes
  9. Moreau-Yosida approximates
  10. Interpolation of smooth min-max games
  11. Co-coercive case
  12. Smooth case
  13. Experiment: showcasing novel constraints
  14. Results and discussion
  15. Experiment's setup description
  16. ...and 3 more sections

Key Result

Theorem 1

Let I be an index set and consider a vector sequence $\left\{(x_i, y_i, g^x_i, g^y_i, f_i)\right\}_{i \in I}$. This sequence is $\mathcal{S}$-interpolable if and only if the following conditions are satisfied: for all $i,j \in I$ the following inequality holds:

Figures (3)

  • Figure 1: Sim-GDA: worst-case contraction rate depending on step size; condition number k = 2
  • Figure 3: Alt-GDA: worst-case contraction rate depending on step size; condition number k = 10
  • Figure 5: Alt-GDA: worst-case contraction rate depending on step size; condition number k = 1000

Theorems & Definitions (53)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Definition 5
  • Definition 6
  • Definition 7
  • Definition 8
  • Definition 9
  • Theorem 1: convex--concave interpolation Krivchenko_Gasnikov_Kovalev_2024
  • ...and 43 more