Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Distributed Learning in Stochastic Non-cooperative Games without Information Exchange

Haidong Li, Anzhi Sheng, Yijie Peng, Long Wang

TL;DR

This work tackles learning Nash equilibria in stochastic non-cooperative games when only noisy black-box evaluations are available and no communication is possible. It introduces a stochastic distributed learning algorithm that combines simultaneous perturbation gradient estimation with mirror descent, enabling players to learn the Nash equilibrium without information exchange. The authors provide a bias-variance analysis of the gradient estimates and prove mean-square convergence for the class of $\beta$-strongly monotone games, achieving convergence rates between $O(n^{-1/2})$ and $O(n^{-1})$ that surpass prior bandit-based methods. Numerical experiments on a Cournot game corroborate faster convergence and robustness to gradient estimation errors, highlighting the method's practical potential for decentralized, black-box settings.

Abstract

In this work, we study stochastic non-cooperative games, where only noisy black-box function evaluations are available to estimate the cost function for each player. Since each player's cost function depends on both its own decision variables and its rivals' decision variables, local information needs to be exchanged through a center/network in most existing work for seeking the Nash equilibrium. We propose a new stochastic distributed learning algorithm that does not require communications among players. The proposed algorithm uses simultaneous perturbation method to estimate the gradient of each cost function, and uses mirror descent method to search for the Nash equilibrium. We provide asymptotic analysis for the bias and variance of gradient estimates, and show the proposed algorithm converges to the Nash equilibrium in mean square for the class of strictly monotone games at a rate faster than the existing algorithms. The effectiveness of the proposed method is buttressed in a numerical experiment.

Efficient Distributed Learning in Stochastic Non-cooperative Games without Information Exchange

TL;DR

This work tackles learning Nash equilibria in stochastic non-cooperative games when only noisy black-box evaluations are available and no communication is possible. It introduces a stochastic distributed learning algorithm that combines simultaneous perturbation gradient estimation with mirror descent, enabling players to learn the Nash equilibrium without information exchange. The authors provide a bias-variance analysis of the gradient estimates and prove mean-square convergence for the class of -strongly monotone games, achieving convergence rates between and that surpass prior bandit-based methods. Numerical experiments on a Cournot game corroborate faster convergence and robustness to gradient estimation errors, highlighting the method's practical potential for decentralized, black-box settings.

Abstract

In this work, we study stochastic non-cooperative games, where only noisy black-box function evaluations are available to estimate the cost function for each player. Since each player's cost function depends on both its own decision variables and its rivals' decision variables, local information needs to be exchanged through a center/network in most existing work for seeking the Nash equilibrium. We propose a new stochastic distributed learning algorithm that does not require communications among players. The proposed algorithm uses simultaneous perturbation method to estimate the gradient of each cost function, and uses mirror descent method to search for the Nash equilibrium. We provide asymptotic analysis for the bias and variance of gradient estimates, and show the proposed algorithm converges to the Nash equilibrium in mean square for the class of strictly monotone games at a rate faster than the existing algorithms. The effectiveness of the proposed method is buttressed in a numerical experiment.
Paper Structure (10 sections, 4 theorems, 50 equations, 2 figures, 1 algorithm)

This paper contains 10 sections, 4 theorems, 50 equations, 2 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem Formulation
  3. Problem Statement
  4. Assumptions
  5. Stochastic Learning
  6. Simultaneous Perturbation
  7. Mirror Descent
  8. Stochastic Distributed Learning Algorithm
  9. Numerical Simulations
  10. Conclusions

Key Result

Theorem 3.1

Suppose $f(x)$ is third-order continuously differentiable, and $\sigma_{f}^{2}(x)$ for any $x\in\prod_{i}X_{i}$ is bounded. Then as $h\to 0$ and $\ell\to\infty$, the order of bias of $\widehat{\nabla}f(x)$ satisfies the order of variance of $\widehat{\nabla}f(x)$ satisfies and thus the order of mean squared error of $\widehat{\nabla}f(x)$ satisfies

Figures (2)

  • Figure 1: Schematic Illustration of Mirror Descent.
  • Figure 2: Convergence to Nash equilibrium.

Theorems & Definitions (12)

  • Definition 2.1
  • Remark 2.2
  • Example 2.3
  • Theorem 3.1
  • proof
  • Remark 3.2
  • Remark 3.3
  • Remark 3.4
  • Theorem 3.5
  • proof
  • ...and 2 more