The Mutual Information In The Vicinity of Capacity-Achieving Input Distributions
Barış Nakiboğlu, Hao-Chung Cheng
TL;DR
This work analyzes how mutual information behaves in the vicinity of capacity-achieving input distributions under polyhedral convex constraints and finite input alphabets. It develops a two-pronged approach: first, a simple, general quadratic decay bound near the capacity-achieving set using Topsøe's identity, Pinsker's inequality, and convex-cone geometry; second, an exact characterization of the slowest decay via Taylor expansions and Moreau's decomposition, yielding explicit leading-term coefficients that determine whether the decay is linear or quadratic in distance to $\\Pi_{\mathscr{A}}$. The results extend to classical-quantum channels, with analogous bounds under separable or finite-dimensional output spaces, and are complemented by counterexamples showing the necessity of the finite-input and polyhedral assumptions. Collectively, the findings provide sharp, non-asymptotic insights into how close input distributions must be to CAID to retain near-capacity mutual information, with potential implications for non-asymptotic coding analyses and quantum-channel robustness. The methods—Topsøe identity, Moreau decomposition, Taylor expansions, and convex-analytic tools—offer a versatile framework for related information-theoretic problems beyond the current setting.
Abstract
The mutual information is bounded from above by a decreasing affine function of the square of the distance between the input distribution and the set of all capacity-achieving input distributions $Π_{\mathcal{A}}$, on small enough neighborhoods of $Π_{\mathcal{A}}$, using an identity due to Topsøe and the Pinsker's inequality, assuming that the input set of the channel is finite and the constraint set $\mathcal{A}$ is polyhedral, i.e., can be described by (possibly multiple but) finitely many linear constraints. Counterexamples demonstrating nonexistence of such a quadratic bound are provided for the case of infinitely many linear constraints and the case of infinite input sets. Using Taylor's theorem with the remainder term, rather than the Pinsker's inequality and invoking Moreau's decomposition theorem the exact characterization of the slowest decrease of the mutual information with the distance to $Π_{\mathcal{A}}$ is determined on small neighborhoods of $Π_{\mathcal{A}}$. Corresponding results for classical-quantum channels are established under separable output Hilbert space assumption for the quadratic bound and under finite-dimensional output Hilbert space assumption for the exact characterization. Implications of these observations for the channel coding problem and applications of the proof techniques to related problems are discussed.