Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Reverse em-problem based on Bregman divergence and its application to classical and quantum information theory

Masahito Hayashi

TL;DR

The paper tackles the problem of computing channel capacity without iteration by reframing the reverse em-problem in a $Bregman$-divergence–driven information-geometric setting. It establishes existence and uniqueness of the inverse map under structured conditions, and shows how the reverse em-problem can be transformed into standard em-problems, leading to a non-iterative formula in special cases and a deeper geometric understanding. By developing a general theory for $m$- and $e$-projections, dual parametrizations, and projections in both classical and quantum contexts, the work extends and unifies analytic capacity calculations previously proposed in the literature. The approach not only clarifies why certain analytic methods succeed in specific instances but also broadens their applicability to classical and quantum channel problems, including wiretap and classical-quantum channels, thereby offering a versatile toolbox for non-iterative capacity computation and information-geometry–driven optimization. Overall, the paper demonstrates that information geometry, via $Bregman$ divergence and projection theory, provides a powerful lens for transforming and solving reverse-EM optimization problems with practical impact on channel capacity analyses.

Abstract

The recent paper (IEEE Trans. IT 69, 1680) introduced an analytical method for calculating the channel capacity without the need for iteration. This method has certain limitations that restrict its applicability. Furthermore, the paper does not provide an explanation as to why the channel capacity can be solved analytically in this particular case. In order to broaden the scope of this method and address its limitations, we turn our attention to the reverse em-problem, proposed by Toyota (Information Geometry, 3, 1355 (2020)). This reverse em-problem involves iteratively applying the inverse map of the em iteration to calculate the channel capacity, which represents the maximum mutual information. However, several open problems remained unresolved in Toyota's work. To overcome these challenges, we formulate the reverse em-problem based on Bregman divergence and provide solutions to these open problems. Building upon these results, we transform the reverse em-problem into em-problems and derive a non-iterative formula for the reverse em-problem. This formula can be viewed as a generalization of the aforementioned analytical calculation method. Importantly, this derivation sheds light on the information geometrical structure underlying this special case. By effectively addressing the limitations of the previous analytical method and providing a deeper understanding of the underlying information geometrical structure, our work significantly expands the applicability of the proposed method for calculating the channel capacity without iteration.

Reverse em-problem based on Bregman divergence and its application to classical and quantum information theory

TL;DR

The paper tackles the problem of computing channel capacity without iteration by reframing the reverse em-problem in a -divergence–driven information-geometric setting. It establishes existence and uniqueness of the inverse map under structured conditions, and shows how the reverse em-problem can be transformed into standard em-problems, leading to a non-iterative formula in special cases and a deeper geometric understanding. By developing a general theory for - and -projections, dual parametrizations, and projections in both classical and quantum contexts, the work extends and unifies analytic capacity calculations previously proposed in the literature. The approach not only clarifies why certain analytic methods succeed in specific instances but also broadens their applicability to classical and quantum channel problems, including wiretap and classical-quantum channels, thereby offering a versatile toolbox for non-iterative capacity computation and information-geometry–driven optimization. Overall, the paper demonstrates that information geometry, via divergence and projection theory, provides a powerful lens for transforming and solving reverse-EM optimization problems with practical impact on channel capacity analyses.

Abstract

The recent paper (IEEE Trans. IT 69, 1680) introduced an analytical method for calculating the channel capacity without the need for iteration. This method has certain limitations that restrict its applicability. Furthermore, the paper does not provide an explanation as to why the channel capacity can be solved analytically in this particular case. In order to broaden the scope of this method and address its limitations, we turn our attention to the reverse em-problem, proposed by Toyota (Information Geometry, 3, 1355 (2020)). This reverse em-problem involves iteratively applying the inverse map of the em iteration to calculate the channel capacity, which represents the maximum mutual information. However, several open problems remained unresolved in Toyota's work. To overcome these challenges, we formulate the reverse em-problem based on Bregman divergence and provide solutions to these open problems. Building upon these results, we transform the reverse em-problem into em-problems and derive a non-iterative formula for the reverse em-problem. This formula can be viewed as a generalization of the aforementioned analytical calculation method. Importantly, this derivation sheds light on the information geometrical structure underlying this special case. By effectively addressing the limitations of the previous analytical method and providing a deeper understanding of the underlying information geometrical structure, our work significantly expands the applicability of the proposed method for calculating the channel capacity without iteration.
Paper Structure (9 sections, 8 theorems, 33 equations, 1 figure, 1 table)

This paper contains 9 sections, 8 theorems, 33 equations, 1 figure, 1 table.

Table of Contents

  1. Introduction
  2. Bregman divergence system
  3. Legendre transform
  4. Exponential subfamily
  5. Mixture subfamily
  6. Bregman Divergence and $m$- and $e$- projections
  7. Evaluation of Bregman divergence without Pythagorean theorem
  8. Examples of Bregman divergence
  9. Classical system

Key Result

proposition thmcounterproposition

Given a vector $(a_j)_{j=1}^l$, we consider an exponential subfamily $\mathcal{E} \subset \Theta$ generated by $l$ vectors $v_1,\ldots, v_l \in \mathbb{R}^d$ at $\theta_0 \in \Theta$, and a mixture subfamily $\mathcal{M} \subset \Theta$ generated by the constraint $\sum_{i=1}^d v^i_j \eta_i(\theta)=

Figures (1)

  • Figure 1: Brief idea of our maximization problem: ${\@fontswitch\mathcal{E}}$ is an exponential family. ${\@fontswitch\mathcal{M}}$ is a mixture family. The solid line expresses the direction of the em-algorithm. The dashed line expresses the direction of the reverse em-algorithm. The pair of $\theta_* \in {\@fontswitch\mathcal{E}}$ and $\theta^*\in {\@fontswitch\mathcal{M}}$ realized the maximum.

Theorems & Definitions (9)

  • definition thmcounterdefinition: Bregman Divergence
  • proposition thmcounterproposition: Pythagorean TheoremAmari-Nagaoka
  • lemma thmcounterlemma
  • lemma thmcounterlemma
  • lemma thmcounterlemma
  • lemma thmcounterlemma
  • lemma thmcounterlemma
  • lemma thmcounterlemma
  • theorem 1