Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Third-order Analysis of Channel Coding in the Small-to-Moderate Deviations Regime

Recep Can Yavas, Victoria Kostina, Michelle Effros

TL;DR

The paper addresses the challenge of accurately approximating the maximum attainable code size $M^*(n,\epsilon)$ in the moderate deviations regime for nonsingular DMCs and Gaussian channels with a power constraint. It introduces channel skewness, a third-order statistic that governs the subleading $Q^{-1}(\epsilon)^2$ term in the MD expansion, and provides tight MD bounds that incorporate this skewness. For Gaussian channels, the authors obtain exact channel skewness by refining Shannon’s random coding bound and Vazquez-Vilar’s bound in the CLT regime, and extend third- and fourth-order analyses to BHT in the MD regime. They also refine results for symmetric channels and derive a fourth-order MD expansion for the type-II error exponent, connecting channel coding performance to BHT asymptotics. Overall, the work yields highly accurate finite-blocklength approximations (e.g., for $n$ in $[100,500]$ and $\epsilon$ down to $10^{-10}$), with practical implications for ultra-reliable communications and related information-theoretic tasks.

Abstract

This paper studies the third-order characteristic of nonsingular discrete memoryless channels and the Gaussian channel with a maximal-power constraint. The third-order term in our expansions employs a new quantity here called the channel skewness, which affects the approximation accuracy more significantly as the error probability decreases. For the Gaussian channel, evaluating Shannon's 1959 random coding bound and Vazquez-Vilar's 2021 meta-converse bound in the central limit theorem (CLT) regime enables exact computation of the channel skewness. For discrete memoryless channels, this work generalizes Moulin's 2017 bounds on the asymptotic expansion of the maximum achievable message set size for nonsingular channels from the CLT regime to include the moderate deviations (MD) regime, thereby refining Altuğ and Wagner's 2014 MD result. For an example binary symmetric channel and most practically important $(n, ε)$ pairs, including $n \in [100, 500]$ and $ε\in [10^{-10}, 10^{-1}]$, an approximation up to the channel skewness is the most accurate among several expansions in the literature. A derivation of the third-order term in the type-II error exponent of binary hypothesis testing in the MD regime is also included; the resulting third-order term is similar to the channel skewness.

Third-order Analysis of Channel Coding in the Small-to-Moderate Deviations Regime

TL;DR

The paper addresses the challenge of accurately approximating the maximum attainable code size in the moderate deviations regime for nonsingular DMCs and Gaussian channels with a power constraint. It introduces channel skewness, a third-order statistic that governs the subleading term in the MD expansion, and provides tight MD bounds that incorporate this skewness. For Gaussian channels, the authors obtain exact channel skewness by refining Shannon’s random coding bound and Vazquez-Vilar’s bound in the CLT regime, and extend third- and fourth-order analyses to BHT in the MD regime. They also refine results for symmetric channels and derive a fourth-order MD expansion for the type-II error exponent, connecting channel coding performance to BHT asymptotics. Overall, the work yields highly accurate finite-blocklength approximations (e.g., for in and down to ), with practical implications for ultra-reliable communications and related information-theoretic tasks.

Abstract

This paper studies the third-order characteristic of nonsingular discrete memoryless channels and the Gaussian channel with a maximal-power constraint. The third-order term in our expansions employs a new quantity here called the channel skewness, which affects the approximation accuracy more significantly as the error probability decreases. For the Gaussian channel, evaluating Shannon's 1959 random coding bound and Vazquez-Vilar's 2021 meta-converse bound in the central limit theorem (CLT) regime enables exact computation of the channel skewness. For discrete memoryless channels, this work generalizes Moulin's 2017 bounds on the asymptotic expansion of the maximum achievable message set size for nonsingular channels from the CLT regime to include the moderate deviations (MD) regime, thereby refining Altuğ and Wagner's 2014 MD result. For an example binary symmetric channel and most practically important pairs, including and , an approximation up to the channel skewness is the most accurate among several expansions in the literature. A derivation of the third-order term in the type-II error exponent of binary hypothesis testing in the MD regime is also included; the resulting third-order term is similar to the channel skewness.
Paper Structure (38 sections, 17 theorems, 241 equations, 3 figures, 1 table)

This paper contains 38 sections, 17 theorems, 241 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Contributions of This Work
  4. Notation and Preliminaries
  5. Notation
  6. Definitions Related to Information Density
  7. Discrete Memoryless Channel
  8. Definitions Related to the Optimal Input Distribution
  9. Singularity of a DMC
  10. Main Results
  11. Nonsingular Channels
  12. Refined Results for Symmetric Channels
  13. Refined Asymptotics of BHT
  14. Gaussian Channel
  15. Proofs of Theorems \ref{['thm:mainAch']} and \ref{['thm:mainConv']}
  16. ...and 23 more sections

Key Result

Theorem 1

Suppose that $\epsilon_n$ is an SMD sequence eq:range and that $P_{Y|X}$ is a nonsingular DMC with $V_{\min} > 0$. It holds that where

Figures (3)

  • Figure 1: Achievable rate vs. average error probability for BSC(0.11): The expansions from Theorems \ref{['thm:mainAch']}--\ref{['thm:refinedAchConv']}, excluding the $O(\cdot)$ terms, are shown for the BSC(0.11) with $\epsilon \in [10^{-10}, 10^{-1}]$ and $n = \{100, 250, 500\}$. The upper and lower boundaries of the shaded region correspond to the non-asymptotic bounds in polyanskiy2010Channel; the CLT approximation that takes $\zeta(n, \epsilon) = \frac{1}{2} \log n$ is from polyanskiy2010thesis; Moulin's results are \ref{['eq:achrho']}--\ref{['eq:convrho']}; the saddlepoint approximation is an achievability bound and is from honda2018 and segura2018.
  • Figure 2: Type-I vs. Type-II error probability for BHT: The expansion from Theorem \ref{['thm:NP']}, excluding the $O(\cdot)$ terms, is shown for $P_i = \mathrm{Bern}(0.6)$, $Q_i = \mathrm{Bern}(0.2)$, $i = 1, \dots, n$, $n \in \{100, 250, 500\}$. Our skewness approximation is compared with the true values obtained by the Neyman-Pearson lemma, the CLT approximation from polyanskiy2010Channel, which consists of the terms up to $\frac{1}{2} \log n$, and the first-order LD approximation from cover.
  • Figure 3: Achievable rate vs. average error probability for a Gaussian channel: The expansions from Theorem \ref{['thm:Gaussian']}, excluding the $O(\cdot)$ term, are shown for the Gaussian channel with $P = 10$, $n = 400$, and $\epsilon \in [10^{-5}, 10^{-3}]$. Shannon's non-asymptotic bounds are from shannon1959Probability; Vazquez-Vilar's non-asymptotic bound is from vazquez2021 where the variance of the auxiliary output distribution is optimized numerically; Shannon's LD approximations are from shannon1959Probability; Polyanskiy et al.'s CLT approximation that takes $\zeta(n, \epsilon, P) = \frac{1}{2} \log n$ is from polyanskiy2010Channeltan2015Third.

Theorems & Definitions (18)

  • Definition 1
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Theorem 5
  • Theorem 6: Petrov petrov1975
  • Lemma 1
  • Theorem 7: Chaganty and Sethuraman chaganty
  • Theorem 8
  • ...and 8 more