Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Decomposition of Differential Game

Nanxiang Zhou, Jing Dong, Yutian Li, Baoxiang Wang

Abstract

To understand the complexity of the dynamic of learning in differential games, we decompose the game into components where the dynamic is well understood. One of the possible tools is Helmholtz's theorem, which can decompose a vector field into a potential and a harmonic component. This has been shown to be effective in finite and normal-form games. However, applying Helmholtz's theorem by connecting it with the Hodge theorem on $\mathbb{R}^n$ (which is the strategy space of differential game) is non-trivial due to the non-compactness of $\mathbb{R}^n$. Bridging the dynamic-strategic disconnect through Hodge/Helmoltz's theorem in differential games is then left as an open problem \cite{letcher2019differentiable}. In this work, we provide two decompositions of differential games to answer this question: the first as an exact scalar potential part, a near vector potential part, and a non-strategic part; the second as a near scalar potential part, an exact vector potential part, and a non-strategic part. We show that scalar potential games coincide with potential games proposed by \cite{monderer1996potential}, where the gradient descent dynamic can successfully find the Nash equilibrium. For the vector potential game, we show that the individual gradient field is divergence-free, in which case the gradient descent dynamic may either be divergent or recurrent.

On the Decomposition of Differential Game

Abstract

To understand the complexity of the dynamic of learning in differential games, we decompose the game into components where the dynamic is well understood. One of the possible tools is Helmholtz's theorem, which can decompose a vector field into a potential and a harmonic component. This has been shown to be effective in finite and normal-form games. However, applying Helmholtz's theorem by connecting it with the Hodge theorem on (which is the strategy space of differential game) is non-trivial due to the non-compactness of . Bridging the dynamic-strategic disconnect through Hodge/Helmoltz's theorem in differential games is then left as an open problem \cite{letcher2019differentiable}. In this work, we provide two decompositions of differential games to answer this question: the first as an exact scalar potential part, a near vector potential part, and a non-strategic part; the second as a near scalar potential part, an exact vector potential part, and a non-strategic part. We show that scalar potential games coincide with potential games proposed by \cite{monderer1996potential}, where the gradient descent dynamic can successfully find the Nash equilibrium. For the vector potential game, we show that the individual gradient field is divergence-free, in which case the gradient descent dynamic may either be divergent or recurrent.

Paper Structure

This paper contains 14 sections, 26 theorems, 44 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Related works
  3. Differential games
  4. Our contributions
  5. Main results
  6. Overview of results
  7. Notations and background
  8. Decomposition of differential Games
  9. The scalar potential game
  10. The near vector potential game
  11. Connection to harmonic and Hamiltonian games
  12. Alternative decomposition
  13. Interpolation of scalar and vector potential games
  14. Conclusion and Future Works

Key Result

Theorem 2.1

The simultaneous gradient $Du = (\nabla_{\omega_1} u_1, \ldots, \nabla_{\omega_M} u_M)$ can be decomposed as where ${\mathcal{P}}$ is the class of exact scalar potential game, ${\mathcal{V}}$ is the class of near vector potential game, $\tilde{{\mathcal{P}}}$ is the class of near scalar potential game and $\tilde{{\mathcal{V}}}$ is the class of exact vector potential game.

Figures (6)

  • Figure 1: Illustration of the example function and its weak derivative.
  • Figure 2: In this two-player potential game, the utility functions are given by $u_1(x,y) = -\frac{1}{2}\|x\|^2 + x^Ty - \|x\|^4$ for player 1 and $u_2(x,y) = -\frac{1}{2}\|y\|^2 + x^Ty - \|y\|^4$ for player 2, where $x,y \in \mathbb{R}^2$. These utilities are derived from the potential function $P(x,y) = -\frac{1}{4}(\|x\|^2 + \|y\|^2) + \frac{1}{2}x^Ty - \frac{1}{4}(\|x\|^4 + \|y\|^4)$. The unique Nash equilibrium is at $(0,0)$.
  • Figure 3: The dynamics of gradient descent, changes of utility over time and gradient vector field of the game presented in Example \ref{['example:orbit']}.
  • Figure 4: The dynamics of gradient descent, changes of utility over time, and gradient vector field of a two-player game with utility gradients $Du = (y,x)$. We plotted the trajectory that started from a set of initial points. We can see that the volume of the set remains unchanged throughout the trajectory.
  • Figure 5: Decomposition of games. The vector Laplacian is defined as $\Delta_1 = d_2^* \circ d_2+d_1 \circ d_1^*,$
  • ...and 1 more figures

Theorems & Definitions (59)

  • Theorem 2.1: Informal statement combining Theorem \ref{['thm:decompose_C1']} and Theorem \ref{['thm:decompose_C1_tilde']}
  • Theorem 2.2: Informal statement combining Theorem \ref{['thm:poincare_near_vector']} and Theorem \ref{['thm:poincare_vector']}
  • Definition 2.1: Weak derivative, evans1998partial
  • Example 2.1
  • Definition 2.2: Sobolev Space, evans1998partial
  • Definition 2.3
  • Definition 2.4
  • Definition 2.5
  • Definition 2.6
  • Definition 2.7
  • ...and 49 more