Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Sparse Superposition Codes with Binomial Dictionary are Capacity-Achieving with Maximum Likelihood Decoding

Yoshinari Takeishi, Jun'ichi Takeuchi

TL;DR

This work demonstrates that sparse superposition codes maintain capacity-achieving performance under maximum likelihood decoding when the dictionary is drawn from a binomial distribution, extending prior results for Gaussian and Bernoulli dictionaries. By constructing a binomial dictionary via summing $d$ Bernoulli components, the authors show that the decoding error probability decays exponentially with block length and that the gap to capacity scales favorably as $d$ grows, due to improved approximation to Gaussian behavior by the Central Limit Theorem. Theoretical bounds are complemented by numerical calculations showing that large $d$ yields achievable rates near the Gaussian-case performance, while the Bernoulli case ($d=1$) exhibits weaker bounds. Overall, the paper advances discrete, hardware-friendly dictionary designs for SS codes and clarifies how increased binomial trials tighten the ML-decoding error bounds, moving closer to the information-theoretic optimum.

Abstract

It is known that sparse superposition codes asymptotically achieve the channel capacity over the additive white Gaussian noise channel with both maximum likelihood decoding and efficient decoding (Joseph and Barron in 2012, 2014). Takeishi et al. (in 2014, 2019) demonstrated that these codes can also asymptotically achieve the channel capacity with maximum likelihood decoding when the dictionary is drawn from a Bernoulli distribution. In this paper, we extend these results by showing that the dictionary distribution can be naturally generalized to the binomial distribution.

Sparse Superposition Codes with Binomial Dictionary are Capacity-Achieving with Maximum Likelihood Decoding

TL;DR

This work demonstrates that sparse superposition codes maintain capacity-achieving performance under maximum likelihood decoding when the dictionary is drawn from a binomial distribution, extending prior results for Gaussian and Bernoulli dictionaries. By constructing a binomial dictionary via summing Bernoulli components, the authors show that the decoding error probability decays exponentially with block length and that the gap to capacity scales favorably as grows, due to improved approximation to Gaussian behavior by the Central Limit Theorem. Theoretical bounds are complemented by numerical calculations showing that large yields achievable rates near the Gaussian-case performance, while the Bernoulli case () exhibits weaker bounds. Overall, the paper advances discrete, hardware-friendly dictionary designs for SS codes and clarifies how increased binomial trials tighten the ML-decoding error bounds, moving closer to the information-theoretic optimum.

Abstract

It is known that sparse superposition codes asymptotically achieve the channel capacity over the additive white Gaussian noise channel with both maximum likelihood decoding and efficient decoding (Joseph and Barron in 2012, 2014). Takeishi et al. (in 2014, 2019) demonstrated that these codes can also asymptotically achieve the channel capacity with maximum likelihood decoding when the dictionary is drawn from a Bernoulli distribution. In this paper, we extend these results by showing that the dictionary distribution can be naturally generalized to the binomial distribution.

Paper Structure

This paper contains 14 sections, 9 theorems, 85 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Review of the original SS codes
  3. Problem Setting
  4. Encoding
  5. Performance of Maximum Likelihood Decoding
  6. SS codes with binomial dictionary
  7. Method to generate dictionary
  8. Review of the Bernoulli dictionary case
  9. Performance of Maximum Likelihood Decoding
  10. Proof of Lemma \ref{['lem2']}
  11. Numerical Calculations
  12. Details of the Calculation
  13. Proof of Lemma \ref{['lem2-1']}
  14. Conclusion

Key Result

Theorem 1

Each element of the dictionary $X$ is independently generated from $N(0, P/L)$. If the section size rate $a \geq a_{v,L}$, then for any transmission rate $R<C$, we have where

Figures (1)

  • Figure 1: Graphs of achievable rates for the original normal distribution dictionary and the binomial distribution dictionary ($d=100$, $d=1000$).

Theorems & Definitions (9)

  • Theorem 1: Joseph and Barron 2012
  • Lemma 1: Joseph and Barron 2012
  • Theorem 2
  • Lemma 2
  • Lemma 3: Takeishi et al. 2014
  • Lemma 4: Takeishi and Takeuchi 2019
  • Lemma 5: Takeishi and Takeuchi 2019
  • Lemma 6: Takeishi and Takeuchi 2019
  • Lemma 7