Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On entropy-constrained Gaussian channel capacity via the moment problem

Adway Girish, Shlomo Shamai, Emre Telatar

TL;DR

This work analyzes the entropy-constrained capacity $C_H(h, \mathsf{snr})$ of the power-constrained Gaussian channel in the low-entropy regime, recasting capacity optimization as a moment-matching problem. The authors establish the existence of a capacity-achieving input and derive tight asymptotics for $C_H$ via the $F_I$ curves, linking $C_H$ to the classical capacity $C(\mathsf{snr})$ in various limits. A central technical contribution is a low-entropy moment problem: for any continuous $W$ with finite moments, a discrete $X$ with entropy below a threshold can match at most three initial moments, with three moments achievable for arbitrarily small entropy. Leveraging this, they show that as $\mathsf{snr}\to 0$ and $h < h_2(1/3)\approx 0.92$ bits, $C_H(h, \mathsf{snr})$ is within a constant factor of $C(\mathsf{snr})$, with a $O(\mathsf{snr}^4)$ gap, highlighting entropy as the key limiter of moment matching and hence of capacity under tight input entropy constraints.

Abstract

We study the capacity of the power-constrained additive Gaussian channel with an entropy constraint at the input. In particular, we characterize this capacity in the low signal-to-noise ratio regime at small entropy. This follows as a corollary of the following general result on a moment matching problem: We show that for any continuous random variable with finite moments, the largest number of initial moments that can be matched by a discrete random variable of sufficiently small but positive entropy is three.

On entropy-constrained Gaussian channel capacity via the moment problem

TL;DR

This work analyzes the entropy-constrained capacity of the power-constrained Gaussian channel in the low-entropy regime, recasting capacity optimization as a moment-matching problem. The authors establish the existence of a capacity-achieving input and derive tight asymptotics for via the curves, linking to the classical capacity in various limits. A central technical contribution is a low-entropy moment problem: for any continuous with finite moments, a discrete with entropy below a threshold can match at most three initial moments, with three moments achievable for arbitrarily small entropy. Leveraging this, they show that as and bits, is within a constant factor of , with a gap, highlighting entropy as the key limiter of moment matching and hence of capacity under tight input entropy constraints.

Abstract

We study the capacity of the power-constrained additive Gaussian channel with an entropy constraint at the input. In particular, we characterize this capacity in the low signal-to-noise ratio regime at small entropy. This follows as a corollary of the following general result on a moment matching problem: We show that for any continuous random variable with finite moments, the largest number of initial moments that can be matched by a discrete random variable of sufficiently small but positive entropy is three.
Paper Structure (13 sections, 5 theorems, 19 equations)

This paper contains 13 sections, 5 theorems, 19 equations.

Table of Contents

  1. Introduction
  2. Prior work
  3. Notation
  4. Asymptotic Characterizations of $C_H$
  5. Existence of capacity-achieving distribution
  6. Entropy asymptotics via $F_I$ curves
  7. Low-SNR asymptotics
  8. The Moment Problem
  9. Low-entropy moment problem
  10. Proof of Theorem 2
  11. Proof of (i):
  12. Proof of (ii):
  13. Discussion and Conclusion

Key Result

Proposition 1

For any $h, \mathsf{snr} > 0$, the entropy-constrained capacity defined in eqn: C_H is given by and a maximizing choice of $X$ is zero mean, has unit variance, and has entropy $h$.

Theorems & Definitions (9)

  • Proposition 1
  • proof
  • Proposition 2
  • proof
  • Theorem 1
  • proof
  • Theorem 2
  • Lemma 1
  • proof