Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Alternating Optimization Approach for Computing $α$-Mutual Information and $α$-Capacity

Akira Kamatsuka, Koki Kazama, Takahiro Yoshida

TL;DR

This work develops variational characterizations for $α$-mutual information and $α$-capacity and uses them to build alternating-optimization (AO) algorithms that iteratively refine reverse-channel and auxiliary distributions. The main contributions include new characterizations for Sibson, Augustin–Csiszár, and Lapidoth–Pfister MI, plus AO algorithms for computing $α$-MI and $α$-capacity across $α>0$; global convergence results and error bounds support their validity. A key finding is that the Sibson MI–based AO exhibits the fastest convergence among the tested schemes, and all AO methods converge to the same $α$-capacity values, consistent with known equivalences among capacity definitions. The results provide practical, provably convergent methods for evaluating $α$-MI and $α$-capacity, with potential applications in error exponents, privacy, and hypothesis testing under Rényi measures.

Abstract

This study presents alternating optimization (AO) algorithms for computing $α$-mutual information ($α$-MI) and $α$-capacity based on variational characterizations of $α$-MI using a reverse channel. Specifically, we derive several variational characterizations of Sibson, Arimoto, Augustin--Csisz{\' a}r, and Lapidoth--Pfister MI and introduce novel AO algorithms for computing $α$-MI and $α$-capacity; their performances for computing $α$-capacity are also compared. The comparison results show that the AO algorithm based on the Sibson MI's characterization has the fastest convergence speed.

Alternating Optimization Approach for Computing $α$-Mutual Information and $α$-Capacity

TL;DR

This work develops variational characterizations for -mutual information and -capacity and uses them to build alternating-optimization (AO) algorithms that iteratively refine reverse-channel and auxiliary distributions. The main contributions include new characterizations for Sibson, Augustin–Csiszár, and Lapidoth–Pfister MI, plus AO algorithms for computing -MI and -capacity across ; global convergence results and error bounds support their validity. A key finding is that the Sibson MI–based AO exhibits the fastest convergence among the tested schemes, and all AO methods converge to the same -capacity values, consistent with known equivalences among capacity definitions. The results provide practical, provably convergent methods for evaluating -MI and -capacity, with potential applications in error exponents, privacy, and hypothesis testing under Rényi measures.

Abstract

This study presents alternating optimization (AO) algorithms for computing -mutual information (-MI) and -capacity based on variational characterizations of -MI using a reverse channel. Specifically, we derive several variational characterizations of Sibson, Arimoto, Augustin--Csisz{\' a}r, and Lapidoth--Pfister MI and introduce novel AO algorithms for computing -MI and -capacity; their performances for computing -capacity are also compared. The comparison results show that the AO algorithm based on the Sibson MI's characterization has the fastest convergence speed.
Paper Structure (20 sections, 20 theorems, 42 equations, 1 figure, 2 tables)

This paper contains 20 sections, 20 theorems, 42 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Variational Characterizations of $\alpha$-MI
  4. AO Algorithms
  5. AO Algorithms for Computing the Augustin--Csiszár MI
  6. AO Algorithms for Computing the Lapidoth--Pfister MI
  7. AO Algorithms for Computing $\alpha$-Capacity
  8. Global Convergence Properties of AO Algorithms
  9. AO algorithms for computing $\alpha$-capacity
  10. AO algorithm for computing Augustin--Csiszár MI
  11. AO algorithms for computing Lapidoth--Pfister MI
  12. Comparison of AO algorithms for $\alpha$-Capacity
  13. Conclusion
  14. Proof of Theorem \ref{['thm:variational_characterization_Sibson_MI']}
  15. Proof of Theorem \ref{['thm:variational_characterization_Csiszar_MI']}
  16. ...and 5 more sections

Key Result

Proposition 1

where the minimum in eq:min_characterization_Shannon_MI is taken over all distributions on $\mathcal{Y}$ and is achieved at $q_{Y}^{*} = p_{Y}$. The minimum in eq:minmin_characterization_Shannon_MI is taken over all product distributions on $\mathcal{X}\times \mathcal{Y}$ and is achieved at $(q_{X}^

Figures (1)

  • Figure 1: Transitions of the approximate value of $\alpha$-capacity $F^{(k)}$ as $k$ increases for (a) $\alpha=1.03$, (b) $\alpha=1.5$, (c) $\alpha=2.0$, and (d) $\alpha=5.0$. The blue solid curve represents Algorithm S1, the orange dashed curve denotes Algorithm JO (modified Jitsumasu--Oohama algorithm), the green dash-dotted curve represents Algorithm C, and the red dash-dot curve indicates Algorithm LP.

Theorems & Definitions (41)

  • Proposition 1: Polyanskiy_Wu_2024
  • Definition 1
  • Remark 1
  • Proposition 2: 370121, 4595361, 6034266, e21080778
  • Theorem 1: arimoto1977,10619200
  • proof
  • Remark 2
  • Theorem 2
  • proof
  • Remark 3
  • ...and 31 more