Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

PAPAL: A Provable PArticle-based Primal-Dual ALgorithm for Mixed Nash Equilibrium

Shihong Ding, Hanze Dong, Cong Fang, Zhouchen Lin, Tong Zhang

TL;DR

PAPAL is the first implementable particle-based algorithm accompanied by non-asymptotic quantitative convergence results, running time, and sample complexity guarantees and is the first implementable particle-based algorithm accompanied by non-asymptotic quantitative convergence results, running time and sample complexity guarantees.

Abstract

We consider the non-convex non-concave objective function in two-player zero-sum continuous games. The existence of pure Nash equilibrium requires stringent conditions, posing a major challenge for this problem. To circumvent this difficulty, we examine the problem of identifying a mixed Nash equilibrium, where strategies are randomized and characterized by probability distributions over continuous domains. To this end, we propose PArticle-based Primal-dual ALgorithm (PAPAL) tailored for a weakly entropy-regularized min-max optimization over probability distributions. This algorithm employs the stochastic movements of particles to represent the updates of random strategies for the $ε$-mixed Nash equilibrium. We offer a comprehensive convergence analysis of the proposed algorithm, demonstrating its effectiveness. In contrast to prior research that attempted to update particle importance without movements, PAPAL is the first implementable particle-based algorithm accompanied by non-asymptotic quantitative convergence results, running time, and sample complexity guarantees. Our framework contributes novel insights into the particle-based algorithms for continuous min-max optimization in the general non-convex non-concave setting.

PAPAL: A Provable PArticle-based Primal-Dual ALgorithm for Mixed Nash Equilibrium

TL;DR

PAPAL is the first implementable particle-based algorithm accompanied by non-asymptotic quantitative convergence results, running time, and sample complexity guarantees and is the first implementable particle-based algorithm accompanied by non-asymptotic quantitative convergence results, running time and sample complexity guarantees.

Abstract

We consider the non-convex non-concave objective function in two-player zero-sum continuous games. The existence of pure Nash equilibrium requires stringent conditions, posing a major challenge for this problem. To circumvent this difficulty, we examine the problem of identifying a mixed Nash equilibrium, where strategies are randomized and characterized by probability distributions over continuous domains. To this end, we propose PArticle-based Primal-dual ALgorithm (PAPAL) tailored for a weakly entropy-regularized min-max optimization over probability distributions. This algorithm employs the stochastic movements of particles to represent the updates of random strategies for the -mixed Nash equilibrium. We offer a comprehensive convergence analysis of the proposed algorithm, demonstrating its effectiveness. In contrast to prior research that attempted to update particle importance without movements, PAPAL is the first implementable particle-based algorithm accompanied by non-asymptotic quantitative convergence results, running time, and sample complexity guarantees. Our framework contributes novel insights into the particle-based algorithms for continuous min-max optimization in the general non-convex non-concave setting.
Paper Structure (39 sections, 27 theorems, 144 equations, 2 figures, 3 algorithms)

This paper contains 39 sections, 27 theorems, 144 equations, 2 figures, 3 algorithms.

Table of Contents

  1. Introduction
  2. Contributions.
  3. Related Work
  4. Min-Max Optimization.
  5. Mixed Nash Equilibrium.
  6. Mean-field Analysis.
  7. Preliminary
  8. Notation.
  9. Problem Setup.
  10. Assumptions.
  11. Primal-Dual Algorithms for Min-Max Optimization.
  12. Unadjusted Langevin Algorithm.
  13. PAPAL: PArticle-based Primal-dual ALgorithm
  14. Exact Primal-dual Method
  15. Recursive Energy Function
  16. ...and 24 more sections

Key Result

Lemma 1

Let $\lambda>0$ be a positive real number and $\tilde{l}(\theta), l(\theta)$ be bounded continuous functions. Consider a probability density $\Bar{p}(\theta)\propto\exp\left(-\tilde{l}(\theta)\right)$, then $p(\theta)\propto\exp\left(-\frac{1}{\lambda+\tau}l(\theta)-\frac{\tau}{\lambda+\tau}\tilde{l

Figures (2)

  • Figure 1: Comparison between weight-driven and particle-based algorithms.
  • Figure 2: Illustration of the sample efficiency on Generative Adversarial Networks. ($x$-axis is the dimension; $y$-axis is the KL divergence to measure the distance between real and fake distribution; $n$ denotes the number of particles)

Theorems & Definitions (32)

  • Definition 1: $\epsilon$-MNE
  • Definition 2
  • Lemma 1
  • Remark 1
  • Remark 2
  • Theorem 1
  • Lemma 2
  • Theorem 2
  • Corollary 1
  • Lemma 3
  • ...and 22 more