Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

New Algorithms for Computing Sibson Capacity and Arimoto Capacity

Akira Kamatsuka, Yuki Ishikawa, Koki Kazama, Takahiro Yoshida

TL;DR

This work advances the computation of Sibson and Arimoto capacities by formulating novel alternating optimization algorithms grounded in $\alpha$-tilted distributions and Hölder's inequality. It develops new variational characterizations and introduces Algorithms S2 and A2 that unify prior approaches (S1/A1) through principled equivalence results under suitable initializations, further proving global convergence. The theoretical contributions are complemented by a numerical example that demonstrates algorithmic equivalence and convergence behavior. The results enhance practical computation of $C_{\alpha}^{\text{S}}$ and $C_{\alpha}^{\text{A}}$ and pave the way for direct computation of the Augustin--Csiszár capacity $C_{\alpha}^{\text{C}}$ in future work.

Abstract

The Sibson and Arimoto capacity, which are based on the Sibson and Arimoto mutual information (MI) of order α, respectively, are well-known generalizations of the channel capacity C. In this study, we derive novel alternating optimization algorithms for computing these capacities by providing new variational characterizations of the Sibson and Arimoto MI. Moreover, we prove that all iterative algorithms for computing these capacities are equivalent under appropriate conditions imposed on their initial distributions.

New Algorithms for Computing Sibson Capacity and Arimoto Capacity

TL;DR

This work advances the computation of Sibson and Arimoto capacities by formulating novel alternating optimization algorithms grounded in -tilted distributions and Hölder's inequality. It develops new variational characterizations and introduces Algorithms S2 and A2 that unify prior approaches (S1/A1) through principled equivalence results under suitable initializations, further proving global convergence. The theoretical contributions are complemented by a numerical example that demonstrates algorithmic equivalence and convergence behavior. The results enhance practical computation of and and pave the way for direct computation of the Augustin--Csiszár capacity in future work.

Abstract

The Sibson and Arimoto capacity, which are based on the Sibson and Arimoto mutual information (MI) of order α, respectively, are well-known generalizations of the channel capacity C. In this study, we derive novel alternating optimization algorithms for computing these capacities by providing new variational characterizations of the Sibson and Arimoto MI. Moreover, we prove that all iterative algorithms for computing these capacities are equivalent under appropriate conditions imposed on their initial distributions.
Paper Structure (13 sections, 10 theorems, 19 equations, 3 figures, 2 tables, 1 algorithm)

This paper contains 13 sections, 10 theorems, 19 equations, 3 figures, 2 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. $\alpha$-mutual information and $\alpha$-capacity
  4. Arimoto--Blahut algorithm
  5. New Algorithms for Calculating Sibson and Arimoto Capacities
  6. Equivalence of algorithms
  7. Equivalence of algorithms
  8. Numerical Example
  9. Conclusion
  10. Proof of Proposition \ref{['prop:properies_alpha_dist']}
  11. Proof of Theorem \ref{['thm:novel_max_characterization']}
  12. Proof of Theorem \ref{['thm:novel_update_formulae']}
  13. Proof of Theorem \ref{['thm:equivalence_S1_S2_A1_A2']}

Key Result

Proposition 1

Let $\alpha, \beta\in (0, 1)\cup (1, \infty)$. Given a distribution $p_{X}$,

Figures (3)

  • Figure 1: Iterations of $p_{X}^{(k)}$ and $q_{X\mid Y}^{(k)}$ in the Arimoto--Blahut algorithm.
  • Figure 2: Iterations of $p_{X}^{(k)}$ and $q_{X\mid Y}^{(k)}$ in Algorithm S1 (solid line) and Algorithm S2 (dashed line) when $p_{X}^{(0), \text{S1}} = p_{X}^{(0), \text{S2}}$.
  • Figure 3: Iterations of $p_{X}^{(k)}$ and $q_{X\mid Y}^{(k)}$ in Algorithm S1 (solid line) and Algorithm A1 (dashed line) when $p_{X}^{(0), \text{S1}} = p_{X_{\alpha}}^{(0), \text{A1}}$.

Theorems & Definitions (26)

  • Definition 1
  • Definition 2
  • Remark 1
  • Proposition 1
  • proof
  • Definition 3
  • Proposition 2: arimoto1977
  • Remark 2
  • Proposition 3: 1054753
  • Proposition 4: arimoto1977,1055640,BN01990060en
  • ...and 16 more