A Symplectic Analysis of Alternating Mirror Descent

TL;DR

The paper investigates Alternating Mirror Descent (AMD) for bilinear zero-sum games through the lens of Hamiltonian dynamics by discretizing the continuous-time flow with the symplectic Euler method. It develops a rigorous framework using Lie algebras and the Baker-Campbell-Hausdorff (BCH) expansion to define a modified Hamiltonian $\widetilde{H}_{\eta}$ that is (formally) conserved by the discretization and that interpolates the AMD trajectory. Key contributions include a closed-form expression for $\widetilde{H}_{\eta}$ in the quadratic case, improved regret bounds $\mathcal{O}(K^{1/5})$ and duality-gap decay $\mathcal{O}(K^{-4/5})$ for AMD, and a conjecture suggesting further improvements to $\mathcal{O}(K^{\varepsilon})$ and $\mathcal{O}(K^{-1+\varepsilon})$ for any $\varepsilon>0$, with absolute convergence yielding $\mathcal{O}(1)$ regret and $\mathcal{O}(K^{-1})$ duality. The work also connects AMD to symplectic integrators via a symmetric decomposition of the payoff matrix, providing a principled energy-based analysis of AMD and proposing avenues for leveraging higher-order symplectic schemes in game-theoretic learning. Overall, the paper bridges symplectic numerical analysis and algorithmic game theory, offering geometric insights and concrete algorithmic guarantees for AMD.

Abstract

Motivated by understanding the behavior of the Alternating Mirror Descent (AMD) algorithm for bilinear zero-sum games, we study the discretization of continuous-time Hamiltonian flow via the symplectic Euler method. We provide a framework for analysis using results from Hamiltonian dynamics, Lie algebra, and symplectic numerical integrators, with an emphasis on the existence and properties of a conserved quantity, the modified Hamiltonian (MH), for the symplectic Euler method. We compute the MH in closed-form when the original Hamiltonian is a quadratic function, and show that it generally differs from the other conserved quantity known previously in that case. We derive new error bounds on the MH when truncated at orders in the stepsize in terms of the number of iterations, $K$, and use these bounds to show an improved $\mathcal{O}(K^{1/5})$ total regret bound and an $\mathcal{O}(K^{-4/5})$ duality gap of the average iterates for AMD. Finally, we propose a conjecture which, if true, would imply that the total regret for AMD scales as $\mathcal{O}\left(K^{\varepsilon}\right)$ and the duality gap of the average iterates as $\mathcal{O}\left(K^{-1+\varepsilon}\right)$ for any $\varepsilon>0$, and we can take $\varepsilon=0$ upon certain convergence conditions for the MH.

Figures (3)

• Figure 1: A simulation of the first 1000 iterations of the algorithm in the case where $F=\log(\cosh(p)), G=\log(\cosh(q))$ and $\eta=0.05$ starting from $p_0=1, q_0=1$. The larger picture on the left contains the trajectory of the iterations, and the smaller four pictures on the right contain the value of the modified Hamiltonian along the iterations. Note that the small label 1e-6+8.5316e-1 (or analogous notation for the other subplots) is Matplotlib’s offset notation: every $y$-value on that axis actually represents $10^{-6}y+0.85316$. In other words, the modified Hamiltonian oscillates a few micro-units around $0.85316$, and that level of oscillation decreases as the truncation order increases.
• Figure 2: The function $\eta \mapsto \frac{\arcsin\left(\sqrt{ab\eta^2}\right)}{\sqrt{ab\eta^2(1-ab\eta^2)}}$ with different choices of $a$ and $b$.
• Figure 3: Trajectories of iterations symplectic Euler discretizations of the Hamiltonian flows plotted in scatter plot. Different colors correspond to different initial positions. (a) $H(x, y) = x^{1.5}+y^{1.5}, \eta=1$ (b) $H(x, y) = x^{2}+y^{2}, \eta=1$ (c) $H(x, y) = x^{4}+y^{4}, \eta=1$

Theorems & Definitions (71)

• Definition 1
• Lemma 1: Wibisono2022
• Lemma 2
• proof
• Lemma 3: Bosch
• Theorem 1
• proof
• Definition 2
• Theorem 2
• Corollary 1