Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalization Bounds for Transformer Channel Decoders

Qinshan Zhang, Bin Chen, Yong Jiang, Shu-Tao Xia

TL;DR

This work addresses the generalization behavior of Transformer-based channel decoders (ECCT) by linking multiplicative-noise estimation to bit-error-rate (BER) and deriving upper bounds on the generalization gap via bit-wise Rademacher complexity. It introduces a theory-grounded framework that first handles a single-layer ECCT and then extends to multi-layer architectures, showing that parity-check–based masked attention reduces the global Lipschitz bound and tightens the bound through reduced hypothesis space complexity. The results reveal how the generalization bound scales with code length, embedding dimension, depth, and training size, and show that sparse masking yields a contraction factor that grows with depth as $\big(\sqrt{P/L}\big)^{T}$. Experiments on AWGN channels with BPSK confirm the predicted trends, supporting the practical relevance of sparsity and depth choices for ECCT design.

Abstract

Transformer channel decoders, such as the Error Correction Code Transformer (ECCT), have shown strong empirical performance in channel decoding, yet their generalization behavior remains theoretically unclear. This paper studies the generalization performance of ECCT from a learning-theoretic perspective. By establishing a connection between multiplicative noise estimation errors and bit-error-rate (BER), we derive an upper bound on the generalization gap via bit-wise Rademacher complexity. The resulting bound characterizes the dependence on code length, model parameters, and training set size, and applies to both single-layer and multi-layer ECCTs. We further show that parity-check-based masked attention induces sparsity that reduces the covering number, leading to a tighter generalization bound. To the best of our knowledge, this work provides the first theoretical generalization guarantees for this class of decoders.

Generalization Bounds for Transformer Channel Decoders

TL;DR

This work addresses the generalization behavior of Transformer-based channel decoders (ECCT) by linking multiplicative-noise estimation to bit-error-rate (BER) and deriving upper bounds on the generalization gap via bit-wise Rademacher complexity. It introduces a theory-grounded framework that first handles a single-layer ECCT and then extends to multi-layer architectures, showing that parity-check–based masked attention reduces the global Lipschitz bound and tightens the bound through reduced hypothesis space complexity. The results reveal how the generalization bound scales with code length, embedding dimension, depth, and training size, and show that sparse masking yields a contraction factor that grows with depth as . Experiments on AWGN channels with BPSK confirm the predicted trends, supporting the practical relevance of sparsity and depth choices for ECCT design.

Abstract

Transformer channel decoders, such as the Error Correction Code Transformer (ECCT), have shown strong empirical performance in channel decoding, yet their generalization behavior remains theoretically unclear. This paper studies the generalization performance of ECCT from a learning-theoretic perspective. By establishing a connection between multiplicative noise estimation errors and bit-error-rate (BER), we derive an upper bound on the generalization gap via bit-wise Rademacher complexity. The resulting bound characterizes the dependence on code length, model parameters, and training set size, and applies to both single-layer and multi-layer ECCTs. We further show that parity-check-based masked attention induces sparsity that reduces the covering number, leading to a tighter generalization bound. To the best of our knowledge, this work provides the first theoretical generalization guarantees for this class of decoders.
Paper Structure (17 sections, 10 theorems, 68 equations, 3 figures, 1 algorithm)

This paper contains 17 sections, 10 theorems, 68 equations, 3 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Problem Statement
  4. Rademacher Complexity and Generalization Bound
  5. Main Results
  6. The Pipeline of ECCT
  7. Generalization Bound via Rademacher Complexity
  8. Generalization Bound for Multi-Layer ECCT
  9. Asymptotic Scaling and Insights
  10. Experiments
  11. Conclusion
  12. Proof of Theorem \ref{['theorem: bound for single-layer ecct (main result)']}
  13. Proof of Theorem \ref{['theorem: impact of mask']}
  14. Gradient Sparsity of Masked Attention
  15. Bounding Lipschitz Constants
  16. ...and 2 more sections

Key Result

Lemma 1

For a real-valued function class $\mathcal{F}$ taking values in $[0,1]$, we have

Figures (3)

  • Figure 1: The pipeline of the ECCT decoder.
  • Figure 2: The mask of the $(7,4)$ Hamming code. For the length $L=7+3=10$ sequence, the first 7 positions represent codeword bits and the last 3 represent parity-check equations. The green box marks index 0 of the sequence, which we focus on; the blue shows its check-node connections, and the orange shows the associated bit-to-bit interactions induced by the parity-check constraints.
  • Figure 3: Generalization gap versus (a) number of attention layers $T$ and (b) input sequence length $L$. The theoretical scaling trend is obtained by substituting the corresponding numerical values.

Theorems & Definitions (24)

  • Definition 1: Empirical Rademacher Complexity
  • Definition 2: Covering Number
  • Lemma 1: See bartlett2017spectrallynormalizedmarginbounds
  • Definition 3: Basic (Single-Layer) ECCT
  • Lemma 2
  • proof : Proof of Lemma \ref{['lemma: equivalence of noise vs codeword BER']}
  • Corollary 1
  • Theorem 1
  • proof : Proof-Sketch of Theorem \ref{['theorem: bound for single-layer ecct (main result)']}
  • Definition 4: Sparse Attention of ECCT
  • ...and 14 more