5G LDPC Linear Transformer for Channel Decoding

TL;DR

This paper addresses the scalability challenge of applying transformer-based decoders to 5G NR LDPC codes by delivering a fully differentiable transformer with linear-time complexity. It achieves this by embedding the parity-check structure into a linear attention mechanism, yielding $O(n)$ scaling and enabling decoding of larger block sizes while maintaining competitive bit error rates relative to one-iteration Belief Propagation. The authors introduce both a Transformer and a Linear Transformer architecture, benchmark them against BP, and demonstrate rapid convergence and reproducibility through the Sionna framework. The work provides a path toward AI-assisted, scalable LDPC decoding for next-generation wireless systems, with potential for iterative transformer decoding and further architectural improvements.

Abstract

This work introduces a novel, fully differentiable linear-time complexity transformer decoder and a transformer decoder to correct 5G New Radio (NR) LDPC. We propose a scalable approach to decode linear block codes with $O(n)$ complexity rather than $O(n^2)$ for regular transformers. The architectures' performances are compared to Belief Propagation (BP), the production-level decoding algorithm used for 5G New Radio (NR) LDPC codes. We achieve bit error rate performance that matches a regular Transformer decoder and surpases one iteration BP, also achieving competitive time performance against BP, even for larger block codes. We utilize Sionna, Nvidia's 5G & 6G physical layer research software, for reproducible results.

Figures (8)

• Figure 1: Transformer Block architecture
• Figure 2: BER comparison for $(k, n) = (13, 26)$.
• Figure 3: BER comparison for $(k, n) = (35, 82)$.
• Figure 4: Time comparison between transformer, linear transformer and belief propagation decoding with differing block sizes of code rate r=0.5.
• Figure 5: $(k, n) = (256, 576)$.