Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The entropic doubling constant and robustness of Gaussian codebooks for additive-noise channels

Lampros Gavalakis, Ioannis Kontoyiannis, Mokshay Madiman

TL;DR

The paper analyzes the entropic doubling constant and its impact on the robustness of Gaussian codebooks for additive-noise channels. It develops entropy comparison inequalities for $h(X+Y)$ when replacing $Y$ by a Gaussian, derives sharp large- and small-doubling bounds, and establishes a multidimensional stability framework for Cramér’s theorem. A key result is a multiplicative channel-capacity bound: for a power-constrained additive-noise channel, $I(X^*;X^*+Z) \ge \frac{\text{snr}}{3\text{snr}+2} C(Z;P)$, with extensions to MACs and MIMO, and improved low-SNR performance relative to prior bounds. Collectively, these results quantify how Gaussian inputs remain near-optimal benchmarks across single-user, MAC, and MIMO settings and provide tools for assessing the design of Gaussian codebooks under entropic considerations.

Abstract

Entropy comparison inequalities are obtained for the differential entropy $h(X+Y)$ of the sum of two independent random vectors $X,Y$, when one is replaced by a Gaussian. For identically distributed random vectors $X,Y$, these are closely related to bounds on the entropic doubling constant, which quantifies the entropy increase when adding an independent copy of a random vector to itself. Consequences of both large and small doubling are explored. For the former, lower bounds are deduced on the entropy increase when adding an independent Gaussian, while for the latter, a qualitative stability result for the entropy power inequality is obtained. In the more general case of non-identically distributed random vectors $X,Y$, a Gaussian comparison inequality with interesting implications for channel coding is established: For additive-noise channels with a power constraint, Gaussian codebooks come within a $\frac{\sf snr}{3{\sf snr}+2}$ factor of capacity. In the low-SNR regime this improves the half-a-bit additive bound of Zamir and Erez (2004). Analogous results are obtained for additive-noise multiple access channels, and for linear, additive-noise MIMO channels.

The entropic doubling constant and robustness of Gaussian codebooks for additive-noise channels

TL;DR

The paper analyzes the entropic doubling constant and its impact on the robustness of Gaussian codebooks for additive-noise channels. It develops entropy comparison inequalities for when replacing by a Gaussian, derives sharp large- and small-doubling bounds, and establishes a multidimensional stability framework for Cramér’s theorem. A key result is a multiplicative channel-capacity bound: for a power-constrained additive-noise channel, , with extensions to MACs and MIMO, and improved low-SNR performance relative to prior bounds. Collectively, these results quantify how Gaussian inputs remain near-optimal benchmarks across single-user, MAC, and MIMO settings and provide tools for assessing the design of Gaussian codebooks under entropic considerations.

Abstract

Entropy comparison inequalities are obtained for the differential entropy of the sum of two independent random vectors , when one is replaced by a Gaussian. For identically distributed random vectors , these are closely related to bounds on the entropic doubling constant, which quantifies the entropy increase when adding an independent copy of a random vector to itself. Consequences of both large and small doubling are explored. For the former, lower bounds are deduced on the entropy increase when adding an independent Gaussian, while for the latter, a qualitative stability result for the entropy power inequality is obtained. In the more general case of non-identically distributed random vectors , a Gaussian comparison inequality with interesting implications for channel coding is established: For additive-noise channels with a power constraint, Gaussian codebooks come within a factor of capacity. In the low-SNR regime this improves the half-a-bit additive bound of Zamir and Erez (2004). Analogous results are obtained for additive-noise multiple access channels, and for linear, additive-noise MIMO channels.
Paper Structure (16 sections, 15 theorems, 76 equations)

This paper contains 16 sections, 15 theorems, 76 equations.

Table of Contents

  1. Introduction
  2. Small and large entropic doubling
  3. The Gaussian as worst-case noise
  4. Robustness of Gaussian channel codes
  5. Ideas and techniques
  6. Preliminaries
  7. Entropy comparison inequalities
  8. Adding Gaussian noise
  9. Large doubling
  10. Small doubling
  11. Channel coding bounds
  12. Robustness of Gaussian inputs
  13. Bibliographical comments
  14. Proof of Theorem \ref{['zamirimprovement']}
  15. Multiple access channels
  16. ...and 1 more sections

Key Result

Theorem 1

Suppose $X$ is a random vector in $\mathbb{R}^d$, and $Z$ is an independent Gaussian random vector with the same covariance matrix as $X$. Then, where $\varphi=\frac{1+\sqrt{5}}{2}>1$ is the golden ratio.

Theorems & Definitions (21)

  • Theorem 1: Large doubling
  • Theorem 2: Small doubling
  • Lemma 2.1
  • Lemma 2.2
  • Theorem 3: Multidimensional stability of Cramér's theorem
  • proof
  • Proposition 1
  • Theorem 4: Approximate generalized Gaussian maximum entropy
  • proof
  • Theorem 5: Large doubling
  • ...and 11 more