Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Universal channel coding for general output alphabet

Masahito Hayashi

TL;DR

This work develops universal channel coding for general output alphabets, including continuous ones, by unifying an exponential-error-decay regime with a second-order, near-capacity regime. The encoder uses a type-based packing approach, while the decoder leverages a Bayesian average (via an α-Rényi Clarke–Barron formula) and information-spectrum methods to remain universal across channel families. It provides explicit lower bounds on the error exponent and a second-order coding-rate expansion, applicable to both discrete and continuous outputs through conditions that handle non-compact parameter spaces. The framework integrates exponential-family channels, Rényi divergences, and perturbation analysis to yield a robust, broadly applicable universal coding theory with practical implications for wireless and MIMO systems. The results unify several threads of prior work on compound channels and universal coding, offering precise performance guarantees in both the exponential and finite-blocklength regimes.

Abstract

We propose two types of universal codes that are suited to two asymptotic regimes when the output alphabet is possibly continuous. The first class has the property that the error probability decays exponentially fast and we identify an explicit lower bound on the error exponent. The other class attains the epsilon-capacity the channel and we also identify the second-order term in the asymptotic expansion. The proposed encoder is essentially based on the packing lemma of the method of types. For the decoder, we first derive a Rényi-relative-entropy version of Clarke and Barron's formula the distance between the true distribution and the Bayesian mixture, which is of independent interest. The universal decoder is stated in terms of this formula and quantities used in the information spectrum method. The methods contained herein allow us to analyze universal codes for channels with continuous and discrete output alphabets in a unified manner, and to analyze their performances in terms of the exponential decay of the error probability and the second-order coding rate.

Universal channel coding for general output alphabet

TL;DR

This work develops universal channel coding for general output alphabets, including continuous ones, by unifying an exponential-error-decay regime with a second-order, near-capacity regime. The encoder uses a type-based packing approach, while the decoder leverages a Bayesian average (via an α-Rényi Clarke–Barron formula) and information-spectrum methods to remain universal across channel families. It provides explicit lower bounds on the error exponent and a second-order coding-rate expansion, applicable to both discrete and continuous outputs through conditions that handle non-compact parameter spaces. The framework integrates exponential-family channels, Rényi divergences, and perturbation analysis to yield a robust, broadly applicable universal coding theory with practical implications for wireless and MIMO systems. The results unify several threads of prior work on compound channels and universal coding, offering precise performance guarantees in both the exponential and finite-blocklength regimes.

Abstract

We propose two types of universal codes that are suited to two asymptotic regimes when the output alphabet is possibly continuous. The first class has the property that the error probability decays exponentially fast and we identify an explicit lower bound on the error exponent. The other class attains the epsilon-capacity the channel and we also identify the second-order term in the asymptotic expansion. The proposed encoder is essentially based on the packing lemma of the method of types. For the decoder, we first derive a Rényi-relative-entropy version of Clarke and Barron's formula the distance between the true distribution and the Bayesian mixture, which is of independent interest. The universal decoder is stated in terms of this formula and quantities used in the information spectrum method. The methods contained herein allow us to analyze universal codes for channels with continuous and discrete output alphabets in a unified manner, and to analyze their performances in terms of the exponential decay of the error probability and the second-order coding rate.

Paper Structure

This paper contains 25 sections, 9 theorems, 101 equations, 1 table.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Information quantities
  4. Exponential family
  5. General assumptions
  6. Main results
  7. Exponential evaluation
  8. Second order evaluation
  9. Method of types and universal encoder
  10. $\alpha$-Rényi-relative-entropy version of Clarke-Barron formula
  11. Universal decoder
  12. Exponential family of channels
  13. General assumption
  14. Error exponent
  15. Second order
  16. ...and 10 more sections

Key Result

Theorem 1

Given real numbers $R$ and $R_1$ with $R_1>R$, a distribution $P$ on ${\cal X}$, and a family of channels $\{\bm{W}_{\bm{\theta}}\}_{\bm{\theta}}$, we assume that the family of channels $\{\bm{W}_{\bm{\theta}}\}_{\bm{\theta}}$ satisfies Condition B, D, or F, and that $s I_{1-s}(P,\bm{W}_{\bm{\theta} for every $\bm{\theta} \in \bm{\Theta}$. $\square$

Theorems & Definitions (22)

  • Example 1: Multinomial distributions
  • Example 2: Poisson distributions
  • Example 3: Gaussian distributions
  • Example 4: Finite set
  • Example 5: Gaussian fading channel
  • Example 6: multi-antenna Gaussian channel
  • Example 7: Poisson channel
  • Example 8
  • Example 9
  • Theorem 1
  • ...and 12 more