Lower Bounds on Mutual Information for Linear Codes Transmitted over Binary Input Channels, and for Information Combining
Uri Erez, Or Ordentlich, Shlomo Shamai
TL;DR
This work strengthens lower bounds on the mutual information for binary-input channels by connecting BSC performance to BEC analyses, particularly when the input is uniform over a shifted linear code. It introduces a main bound of the form $I_{ ext{BSC}}^{(t)}(X^n;Y^n) \ge n\bar{\psi}_t\big(I_{ ext{BEC}}^{(\eta_t)}(X^n;Y^n)/(n\eta_t)\big)$ for linear-code inputs, with $\eta_t=(1-2h^{-1}(1-t))^2$ and $\bar{\psi}_t$ the concave envelope of a Mrs. Gerber-type function, improving prior results. Beyond this, the paper provides a general lower bound for arbitrary $P_X$ and $W$ via an input-dependent SDPI coefficient, $I(X;Y^n) \ge \alpha(1-(1-\eta)^n)H(P_X)$, and derives universal bounds for Bernoulli inputs through the BSC, as well as applications to information combining. The results enable bounding BSC mutual information using BEC computations and have concrete implications for the design of information-capacity achieving codes over binary channels. Overall, the work advances the toolkit for analyzing mutual information through channel comparisons and code-structure-specific bounds.
Abstract
It has been known for a long time that the mutual information between the input sequence and output of a binary symmetric channel (BSC) is upper bounded by the mutual information between the same input sequence and the output of a binary erasure channel (BEC) with the same capacity. Recently, Samorodintsky discovered that one may also lower bound the BSC mutual information in terms of the mutual information between the same input sequence and a more capable BEC. In this paper, we strengthen Samordnitsky's bound for the special case where the input to the channel is distributed uniformly over a linear code. Furthermore, for a general (not necessarily binary) input distribution $P_X$ and channel $W_{Y|X}$, we derive a new lower bound on the mutual information $I(X;Y^n)$ for $n$ transmissions of $X\sim P_X$ through the channel $W_{Y|X}$.