Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

High signal-to-noise ratio asymptotics of entropy-constrained Gaussian channel capacity

Adway Girish, Shlomo Shamai, Emre Telatar

Abstract

We study the input-entropy-constrained Gaussian channel capacity problem in the asymptotic high signal-to-noise ratio (SNR) regime. We show that the capacity-achieving distribution as SNR goes to infinity is given by a discrete Gaussian distribution supported on a scaled integer lattice. Further, we show that the gap between the input entropy and the capacity decreases to zero exponentially in SNR, and characterize this exponent.

High signal-to-noise ratio asymptotics of entropy-constrained Gaussian channel capacity

Abstract

We study the input-entropy-constrained Gaussian channel capacity problem in the asymptotic high signal-to-noise ratio (SNR) regime. We show that the capacity-achieving distribution as SNR goes to infinity is given by a discrete Gaussian distribution supported on a scaled integer lattice. Further, we show that the gap between the input entropy and the capacity decreases to zero exponentially in SNR, and characterize this exponent.
Paper Structure (11 sections, 6 theorems, 38 equations, 5 figures)

This paper contains 11 sections, 6 theorems, 38 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Prior work
  3. Notation
  4. Main results
  5. Technical details
  6. Reduction to minimum distance between atoms
  7. (a) Lower bound to $H(X \mid Y)$.
  8. (b) Upper bound to $H(X \mid Y)$.
  9. Minimum distance with entropy constraint
  10. Low- and high-entropy regimes
  11. Discussion and conclusion

Key Result

Theorem 1

For finite $h > 0$, the distribution achieving capacity as $\mathsf{snr} \to \infty$ is $\mathcal{N}_{\mathsf{d}_h \mathbb{Z}}(\lambda_h)$, where $\lambda_h$ is the value of $\lambda$ that solves $h = L(\lambda) - \lambda L'(\lambda)$ and $\mathsf{d}_h = 1/\sqrt{-L'(\lambda_h)}$. Further, the capaci

Figures (5)

  • Figure 1: Figure showing $H(X \mid Y)$ for $X \sim \mathcal{N}_{\mathsf{d}_h \mathbb{Z}}(\lambda_h)$ versus $\mathsf{snr}$, compared against $\exp(-\mathsf{snr} \frac{\mathsf{d}_h^2}{8})$ from Theorem \ref{['thm: char']} at three different values of $h$ (in bits): 0.05, 0.5, 5. For the low and high entropy cases, we also compare against the explicit approximations from Corollary \ref{['cor: ent_asymp']}. $y$-axis in log-scale.
  • Figure 2: Comparison of the capacity-achieving distributions as $\mathsf{snr} \to \infty$ for two example distributions with (a) "small" entropy, $h = 10^{-4}$ bits: approximately a three-point symmetric distribution with a large mass at the origin (note that the $y$-axis is in log-scale) and (b) "large" entropy, $h = 6$: approximately the standard Gaussian distribution.
  • Figure 3: $L(\lambda) = \log(\sum_{i \in \mathbb{Z}} \exp(-\lambda i^2))$ versus $\lambda$. The $x$- and $y$-intercepts of the tangents to $L$ are $h \mathsf{d}_h^2$ and $h$ respectively.
  • Figure 4: $L_a(\lambda)$ versus $\lambda$ for $a = 0$ and $a = \frac{1}{2}$. The $x$- and $y$-intercepts of the tangents to $L_0$ are $h \mathsf{d}_h^2$ and $h$ respectively.
  • Figure 5: Comparison of $\mathsf{d}_h$ and its approximations in the small and large entropy regimes: (a) $\mathsf{d}_h \approx \sqrt{\frac{1}{h}\log\frac{2}{h}}$ for small $h \ll 0.48$ bits, and (b) $\mathsf{d}_h \approx \sqrt{2\pi e} \exp(-h)$ for large $h \gg 0.48$ bits.

Theorems & Definitions (13)

  • Theorem 1: High-SNR asymptotics of $C_H$
  • proof
  • Corollary 1: Low- and high-entropy approximation of $C_H$ at asymptotic high SNR
  • proof
  • Lemma 1: Exponent of conditional entropy
  • proof : Proof
  • Proposition 1: Characterizing distribution with largest $\mathsf{d}_{\min}$
  • proof
  • Lemma 2: Tangents with same $y$-intercept to convex functions
  • proof
  • ...and 3 more