Curved representational Bregman divergences and their applications

TL;DR

This work extends the Bregman divergence framework by introducing curved Bregman divergences, defined on nonlinear subspaces of the parameter space, and showing that their barycenters correspond to right Bregman projections. It develops concrete instances (symmetrized divergences, CN KL, curved simplex KL) and extends to sub-dimensional and representational forms, including α-divergences mapped into positive measure space. A key contribution is the representation-based treatment of α-divergences, enabling efficient computation of intersections of α-spheres and linking to Jeffreys-type centroids via curved geometry. The results unify curved exponential-family intuition with practical centroid computation and geometric tools in divergence-based analysis.

Abstract

By analogy to the terminology of curved exponential families in statistics, we define curved Bregman divergences as Bregman divergences restricted to nonlinear parameter subspaces and sub-dimensional Bregman divergences when the restrictions are linear. A common example of curved Bregman divergence is the cosine dissimilarity between normalized vectors. We show that the barycenter of a finite weighted set of parameters under a curved Bregman divergence amounts to the right Bregman projection onto the nonlinear subspace of the barycenter with respect to the full Bregman divergence. We demonstrate the significance of curved Bregman divergences with two examples: (1) symmetrized Bregman divergences, (2) pointwise symmetrized Bregman divergences, and (3) the Kullback-Leibler divergence between circular complex normal distributions. We explain how to reparameterize sub-dimensional Bregman divergences on simplicial sub-dimensional domains. We then consider monotonic embeddings to define representational curved Bregman divergences and show that the $α$-divergences are representational curved Bregman divergences with respect to $α$-embeddings of the probability simplex into the positive measure cone. As an application, we report an efficient method to calculate the intersection of a finite set of $α$-divergence spheres.

Figures (10)

• Figure 1: Curved exponential family.
• Figure 2: Example of a curved Bregman divergence: The curved circle Euclidean divergence amounts to the cosine dissimilarity.
• Figure 3: Illustration of the reparameterization and sub-dimensional Bregman divergences.
• Figure 4: The discrete Kullback-Leibler divergence between two probability mass functions of the 2D simplex $\Delta_2$ sitting in $\mathbb{R}_{>0}^3$ is an example of sub-dimensional Bregman divergence of the extended KLD defined between positive arrays of ${\tilde{\Theta}}=\mathbb{R}_{>0}^3$.
• Figure 5: The curved Bregman centroid $\theta(\bar{u})$ amounts to the right Bregman projection of the unconstrained Bregman centroid $\bar{\theta}$ onto the subspace $\mathcal{U}$.

Theorems & Definitions (22)

• Definition 1: Curved exponential family, Figure \ref{['fig:curvedExpFam']}
• Example 1: shima2000geometry
• Definition 2: Curved Bregman divergence
• Example 2: curved circle Euclidean divergence
• Example 3
• Proposition 1
• Proposition 2: Reparameterized/sub-dimensional Bregman divergence
• Example 4: KLD as a sub-dimensional Bregman divergence
• Theorem 1
• Example 5