Second-Order Mirror Descent: Convergence in Games Beyond Averaging and Discounting

TL;DR

It is shown that MD2 enjoys no-regret as well as an exponential rate of convergence toward strong VSS upon a slight modification, and stochastic approximation techniques are used to provide a convergence guarantee of discrete-time MD2 with noisy observations toward interior mere VSS.

Abstract

In this paper, we propose a second-order extension of the continuous-time game-theoretic mirror descent (MD) dynamics, referred to as MD2, which provably converges to mere (but not necessarily strict) variationally stable states (VSS) without using common auxiliary techniques such as time-averaging or discounting. We show that MD2 enjoys no-regret as well as an exponential rate of convergence towards strong VSS upon a slight modification. MD2 can also be used to derive many novel continuous-time primal-space dynamics. We then use stochastic approximation techniques to provide a convergence guarantee of discrete-time MD2 with noisy observations towards interior mere VSS. Selected simulations are provided to illustrate our results.

Figures (12)

• Figure 1: $\varsigma = 1$, $\mathcal{G}$ merely monotone, $x^\star$ is merely VS. MD2 converges (Embedded: MD, cycling).
• Figure 2: $\varsigma = 1.1$, $\mathcal{G}$$0.1$-weakly monotone, $x^\star$ is $0.1$-weakly VS. MD2 converges (Embedded: MD, cycling).
• Figure 3: $\varsigma = 1.1$, $\mathcal{G}$$0.1$-weakly monotone, $x^\star$ is $0.1$-weakly VS. MD2 converges (Embedded: MDA, cycling).
• Figure 4: $\varsigma = 1, \sigma_\zeta^2 = 1$, $t_k = 4/k, \tau_k = 1/k$.
• Figure 5: $\varsigma = 1, \sigma^2_\zeta = 10$, $t_k = 0.23/k^{0.48}, \tau_k = 0.34/k^{0.88}$.

Theorems & Definitions (46)

• Remark 1
• Definition 1
• Definition 2
• Remark 2
• Proposition 1
• Remark 3
• Remark 4
• Example 1
• Example 2
• Example 3