Hybrid Mamba-Transformer Decoder for Error-Correcting Codes

TL;DR

The paper tackles decoding of binary linear block codes with deep learning, aiming to improve both accuracy and latency. It introduces ECCM, a hybrid Mamba–Transformer decoder that uses parity-check-aware masks and progressive supervision to guide learning across decoding depths. Experiments across BCH, Polar, LDPC, and MacKay codes show substantial BER improvements on BCH/Polar and competitive results on LDPC, while achieving faster inference than prior neural decoders. The work demonstrates that combining efficient sequential models (Mamba) with global attention (Transformer) and structured masking yields scalable, real-time decoding for diverse code families.

Abstract

We introduce a novel deep learning method for decoding error correction codes based on the Mamba architecture, enhanced with Transformer layers. Our approach proposes a hybrid decoder that leverages Mamba's efficient sequential modeling while maintaining the global context capabilities of Transformers. To further improve performance, we design a novel layer-wise masking strategy applied to each Mamba layer, allowing selective attention to relevant code features at different depths. Additionally, we introduce a progressive layer-wise loss, supervising the network at intermediate stages and promoting robust feature extraction throughout the decoding process. Comprehensive experiments across a range of linear codes demonstrate that our method significantly outperforms Transformer-only decoders and standard Mamba models.

Figures (5)

• Figure 1: Model Blocks
• Figure 2: Mamba block structure
• Figure 3: Model output processing
• Figure 4: Attention block structure
• Figure 6: BER-SNR performance of ECCM versus baselines, on BCH and POLAR codes