Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Computing Augustin Information via Hybrid Geodesically Convex Optimization

Guan-Ren Wang, Chung-En Tsai, Hao-Chung Cheng, Yen-Huan Li

TL;DR

The paper tackles computing the order-$\alpha$ Augustin information $I_\alpha(P_X, P_{Y|X}) = \min_{Q_Y} \mathbb{E}_X D_\alpha(P_{Y|X}(\cdot|X) \| Q_Y)$ by embedding the problem in a Riemannian framework with the Poincaré metric. It introduces a hybrid analysis where the objective is geodesically convex under the Euclidean structure but geodesically smooth under the Poincaré metric, and proves a non-asymptotic $\mathcal{O}(1/T)$ convergence rate for Riemannian gradient descent with step $\eta=1/L$. The approach yields the first finite-time guarantee for all $\alpha>0$ and is validated numerically, showing empirical efficiency relative to fixed-point and other gradient-based methods. The results open pathways to extending the framework to quantum Rényi divergences and other problems with similar geometric structures, enabling robust non-asymptotic guarantees in geodesically non-Euclidean settings.

Abstract

We propose a Riemannian gradient descent with the Poincaré metric to compute the order-$α$ Augustin information, a widely used quantity for characterizing exponential error behaviors in information theory. We prove that the algorithm converges to the optimum at a rate of $\mathcal{O}(1 / T)$. As far as we know, this is the first algorithm with a non-asymptotic optimization error guarantee for all positive orders. Numerical experimental results demonstrate the empirical efficiency of the algorithm. Our result is based on a novel hybrid analysis of Riemannian gradient descent for functions that are geodesically convex in a Riemannian metric and geodesically smooth in another.

Computing Augustin Information via Hybrid Geodesically Convex Optimization

TL;DR

The paper tackles computing the order- Augustin information by embedding the problem in a Riemannian framework with the Poincaré metric. It introduces a hybrid analysis where the objective is geodesically convex under the Euclidean structure but geodesically smooth under the Poincaré metric, and proves a non-asymptotic convergence rate for Riemannian gradient descent with step . The approach yields the first finite-time guarantee for all and is validated numerically, showing empirical efficiency relative to fixed-point and other gradient-based methods. The results open pathways to extending the framework to quantum Rényi divergences and other problems with similar geometric structures, enabling robust non-asymptotic guarantees in geodesically non-Euclidean settings.

Abstract

We propose a Riemannian gradient descent with the Poincaré metric to compute the order- Augustin information, a widely used quantity for characterizing exponential error behaviors in information theory. We prove that the algorithm converges to the optimum at a rate of . As far as we know, this is the first algorithm with a non-asymptotic optimization error guarantee for all positive orders. Numerical experimental results demonstrate the empirical efficiency of the algorithm. Our result is based on a novel hybrid analysis of Riemannian gradient descent for functions that are geodesically convex in a Riemannian metric and geodesically smooth in another.
Paper Structure (24 sections, 12 theorems, 59 equations, 2 figures)

This paper contains 24 sections, 12 theorems, 59 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Computing the Augustin Information
  4. Hybrid Analysis
  5. Preliminaries
  6. Smooth Convex Optimization
  7. Basics of Riemannian Geometry
  8. Geodesically Convex Optimization
  9. Hybrid Analysis of Geodesically Smooth Convex Optimization
  10. Application: Computing Augustin Information
  11. Poincaré Metric
  12. Geodesic Smoothness of the Objective Function
  13. Numerical Results
  14. Conclusion
  15. Proof of Lemma \ref{['lemma:asymptotic convergence']}
  16. ...and 9 more sections

Key Result

Theorem 3.3

If the function to be minimized is g-convex and g-$L$-smooth on a Riemannian manifold $(\mathcal{M}, \mathfrak{g})$, then RGD with step size $\eta=1/L$ converges at a rate of $\mathcal{O}(1/T)$.

Figures (2)

  • Figure 1: Convergence speeds for computing the order-$3$ Augustin information.
  • Figure 2: Slower convergence speeds for the methods proposed by li2019convergence and by you2022minimizing.

Theorems & Definitions (16)

  • Definition 3.1
  • Definition 3.2: Geodesic convexity, geodesic smoothness zhang2016first
  • Theorem 3.3: zhang2016first
  • Lemma 4.2: boumal2023introduction
  • Lemma 4.3: boumal2023introduction
  • Corollary 4.4
  • Lemma 4.5
  • Theorem 4.6
  • Remark 4.7
  • Proposition 5.1: bhatia2019burespennec2006riemannianpennec2020manifold
  • ...and 6 more