Unified Error Correction Code Transformer with Low Complexity

TL;DR

This work tackles the hardware cost and scalability limits of decoding multiple linear block codes in 6G by introducing a Unified Error Correction Code Transformer (UECCT). It replaces the quadratic self-attention of prior AI decoders with a low-rank, shared Unified Attention Module that operates in $O(N)$ time, aided by a sparse mask derived from the parity-check matrix to enforce code constraints. The approach uses a standardized unit to align code lengths and a sparsity-aware mechanism to enable cross-code generalization, achieving up to $86\%$ reduction in attention complexity and improved decoding accuracy across LDPC, Polar, BCH, and other codes relative to ECCT and FECCT. Experiments show that UECCT delivers competitive or superior bit- and block-error performance while significantly reducing training and inference time, indicating strong potential for scalable, low-cost decoders in next-generation wireless systems.

Abstract

Channel coding is vital for reliable sixth-generation (6G) data transmission, employing diverse error correction codes for various application scenarios. Traditional decoders require dedicated hardware for each code, leading to high hardware costs. Recently, artificial intelligence (AI)-driven approaches, such as the error correction code Transformer (ECCT) and its enhanced version, the foundation error correction code Transformer (FECCT), have been proposed to reduce the hardware cost by leveraging the Transformer to decode multiple codes. However, their excessively high computational complexity of $\mathcal{O}(N^2)$ due to the self-attention mechanism in the Transformer limits scalability, where $N$ represents the sequence length. To reduce computational complexity, we propose a unified Transformer-based decoder that handles multiple linear block codes within a single framework. Specifically, a standardized unit is employed to align code length and code rate across different code types, while a redesigned low-rank unified attention module, with computational complexity of $\mathcal{O}(N)$, is shared across various heads in the Transformer. Additionally, a sparse mask, derived from the parity-check matrix's sparsity, is introduced to enhance the decoder's ability to capture inherent constraints between information and parity-check bits, improving decoding accuracy and further reducing computational complexity by $86\%$. Extensive experimental results demonstrate that the proposed unified Transformer-based decoder outperforms existing methods and provides a high-performance, low-complexity solution for next-generation wireless communication systems.

Figures (16)

• Figure 1: Single-layer vanilla Transformer encoder architecture.
• Figure 2: Architecture of multi-head self-attention.
• Figure 3: Architecture of the communication system.
• Figure 4: Decoder framework with pre- and post-processing.
• Figure 5: Rank distribution of the $\textbf{Q} \cdot \textbf{K}^T$ matrix in the ECCT model across 6 encoder layers for various code types.