Linear Convergence in Hilbert's Projective Metric for Computing Augustin Information and a Rényi Information Measure

TL;DR

The paper addresses non-closed-form problems of computing order-$\alpha$ Augustin information and a Rényi information measure of independence by framing the corresponding iterative updates as positive dynamical systems on probability simplices and analyzing them under Hilbert's projective metric. It proves linear convergence for Augustin's fixed-point iteration when $\alpha\in(1/2,1)\cup(1,3/2)$ with rate $O((2|1-\alpha|)^t)$ and for Kamatsuka2024's alternating-minimization algorithm when $\alpha\in[1/2,1)\cup(1,\infty)$ with rate $O(|1-1/\alpha|^{2t})$, improving upon prior asymptotic guarantees. The results rely on contraction bounds for the corresponding operators and, in the full-support case, on an enhanced contraction constant via Birkhoff's theorem. This linear-rate analysis broadens the theoretical understanding of these algorithms and informs practical choices of $\alpha$ and initialization in finite-time computations.

Abstract

Consider the problems of computing the Augustin information and a Rényi information measure of statistical independence, previously explored by Lapidoth and Pfister (IEEE Information Theory Workshop, 2018) and Tomamichel and Hayashi (IEEE Trans. Inf. Theory, 64(2):1064--1082, 2018). Both quantities are defined as solutions to optimization problems and lack closed-form expressions. This paper analyzes two iterative algorithms: Augustin's fixed-point iteration for computing the Augustin information, and the algorithm by Kamatsuka et al. (arXiv:2404.10950) for the Rényi information measure. Previously, it was only known that these algorithms converge asymptotically. We establish the linear convergence of Augustin's algorithm for the Augustin information of order $α\in (1/2, 1) \cup (1, 3/2)$ and Kamatsuka et al.'s algorithm for the Rényi information measure of order $α\in [1/2, 1) \cup (1, \infty)$, using Hilbert's projective metric.

Theorems & Definitions (17)

• Definition 1
• Lemma 2
• Lemma 3
• Lemma 4
• Theorem 5: Birkhoff1957
• Lemma 6: Nakiboglu2019
• Theorem 7
• Corollary 8
• Remark 9
• Lemma 10