Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Symplectic Bregman divergences

Frank Nielsen

TL;DR

This work generalizes Bregman divergences to symplectic vector spaces by formulating a symplectic Fenchel-Young framework and defining a symplectic gradient via the symplectic subdifferential. Divergences $B_F^{\omega}(z_1:z_2)$ are built from a convex potential $F$ and its symplectic conjugate $F^{*\omega}$, guaranteeing nonnegativity and reducing to ordinary Bregman divergences when the symplectic form stems from an inner product. The construction recovers standard BD with a composite inner product in the appropriate limit and extends BD notions to dual systems through a pairing-based symplectic form. The framework connects to geometric mechanics and learning dynamics, suggesting applications in information geometry and dual-system analysis, with perspective-transform and proximation interpretations discussed as part of the broader theoretical landscape.

Abstract

We present a generalization of Bregman divergences in symplectic vector spaces that we term symplectic Bregman divergences. Symplectic Bregman divergences are derived from a symplectic generalization of the Fenchel-Young inequality which relies on the notion of symplectic subdifferentials. The symplectic Fenchel-Young inequality is obtained using the symplectic Fenchel transform which is defined with respect to the symplectic form. Since symplectic forms can be generically built from pairings of dual systems, we get a generalization of Bregman divergences in dual systems obtained by equivalent symplectic Bregman divergences. In particular, when the symplectic form is derived from an inner product, we show that the corresponding symplectic Bregman divergences amount to ordinary Bregman divergences with respect to composite inner products. Some potential applications of symplectic divergences in geometric mechanics, information geometry, and learning dynamics in machine learning are touched upon.

Symplectic Bregman divergences

TL;DR

This work generalizes Bregman divergences to symplectic vector spaces by formulating a symplectic Fenchel-Young framework and defining a symplectic gradient via the symplectic subdifferential. Divergences are built from a convex potential and its symplectic conjugate , guaranteeing nonnegativity and reducing to ordinary Bregman divergences when the symplectic form stems from an inner product. The construction recovers standard BD with a composite inner product in the appropriate limit and extends BD notions to dual systems through a pairing-based symplectic form. The framework connects to geometric mechanics and learning dynamics, suggesting applications in information geometry and dual-system analysis, with perspective-transform and proximation interpretations discussed as part of the broader theoretical landscape.

Abstract

We present a generalization of Bregman divergences in symplectic vector spaces that we term symplectic Bregman divergences. Symplectic Bregman divergences are derived from a symplectic generalization of the Fenchel-Young inequality which relies on the notion of symplectic subdifferentials. The symplectic Fenchel-Young inequality is obtained using the symplectic Fenchel transform which is defined with respect to the symplectic form. Since symplectic forms can be generically built from pairings of dual systems, we get a generalization of Bregman divergences in dual systems obtained by equivalent symplectic Bregman divergences. In particular, when the symplectic form is derived from an inner product, we show that the corresponding symplectic Bregman divergences amount to ordinary Bregman divergences with respect to composite inner products. Some potential applications of symplectic divergences in geometric mechanics, information geometry, and learning dynamics in machine learning are touched upon.
Paper Structure (8 sections, 1 theorem, 29 equations, 2 figures)

This paper contains 8 sections, 1 theorem, 29 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Dual systems, symplectic forms, and symplectomorphisms
  3. Symplectic forms derived from dual systems
  4. Linear symplectomorphisms and the groups of symplectic matrices
  5. Symplectic Fenchel transform, symplectic subdifferentials, and symplectic Fenchel-Young (in)equality
  6. Symplectic Fenchel-Young divergences and symplectic Bregman divergences
  7. Particular cases recover composite Bregman divergences
  8. Summary, discussion, and perspectives

Key Result

Theorem 1

Let $F(z)$ be a convex (i.e., $F(z)=F(x,y)$ is joint convex, i.e., convex with respect to $z=(x,y)$) and lower semicontinuous function. Then the following inequality holds: with equality if and only if $z'\in\partial^\omega(z)$.

Figures (2)

  • Figure 1: The motion of a single point particle $q(t)$ with mass $m$ and momentum $p(t)=m\dot{q}(t)$ on a 1D line can be modeled as a curve $\mathcal{C}=\{c(t)=(q(t),p(t)) {\ :\ } t\in T\subset\mathbb{R}\}$ in the phase space $\mathbb{R}^2$.
  • Figure 3: Bregman divergences generalized to dual systems $(X,Y,{b(\cdot,\cdot)})$: A symplectic form $\omega$ on the space $Z=X\oplus Z$ is induced by the pairing product. The Bregman divergence on the dual system is then defined as the symplectic Bregman divergence on the symplectic vector space $(Z,\omega)$.

Theorems & Definitions (9)

  • Definition 1: Dual system.
  • Definition 2: Symplectic vector space
  • Remark 1
  • Remark 2
  • Theorem 1: Symplectic Fenchel-Young inequality, Theorem 2.3 of BuligaSaxce-2017
  • Definition 3: Symplectic Fenchel-Young divergence
  • Definition 4: Symplectic Bregman divergence
  • Remark 3
  • Definition 5: SBEN principle BuligaSaxce-2017