LightCode: Light Analytical and Neural Codes for Channels with Feedback

TL;DR

This work demonstrates that Power Blast, an analytical coding scheme inspired by Schalkwijk-Kailath and Gallager-Nakiboğlu schemes, achieves notable reliability improvements over both SK and GN schemes, and proposes Light Code, a lightweight neural code that achieves state-of-the-art reliability while using a fraction of memory and compute compared to existing deep-learning-based codes.

Abstract

The design of reliable and efficient codes for channels with feedback remains a longstanding challenge in communication theory. While significant improvements have been achieved by leveraging deep learning techniques, neural codes often suffer from high computational costs, a lack of interpretability, and limited practicality in resource-constrained settings. We focus on designing low-complexity coding schemes that are interpretable and more suitable for communication systems. We advance both analytical and neural codes. First, we demonstrate that PowerBlast, an analytical coding scheme inspired by Schalkwijk-Kailath (SK) and Gallager-Nakiboğlu (GN) schemes, achieves notable reliability improvements over both SK and GN schemes, outperforming neural codes in high signal-to-noise ratio (SNR) regions. Next, to enhance reliability in low-SNR regions, we propose LightCode, a lightweight neural code that achieves state-of-the-art reliability while using a fraction of memory and compute compared to existing deeplearning-based codes. Finally, we systematically analyze the learned codes, establishing connections between LightCode and PowerBlast, identifying components crucial for performance, and providing interpretation aided by linear regression analysis.

Figures (12)

• Figure 1: Illustration of the $i^{\text{th}}$ round of communication for channels with feedback. The encoder takes as input the message bits $\mathbf{u}$ and the encoder output from previous rounds $x^{(i-1)}$, concatenated with the feedback from previous rounds $\Tilde{y}^{(i-1)}$, to compute $x_i$.
• Figure 2: By combining the SK and discrete-symbol strategy of the GN scheme, PowerBlast noticeably improves the BLER performance upon both SK and GN schemes.
• Figure 3: (Left)Architecture for GBAF: The positional encoding and transformer encoding modules are used for block coding to encourage the mixing of symbols across the positions. (Right): Using a symbol-by-symbol scheme, LightCode significantly reduces the complexity of encoding and achieves more than $10$x reduction in the number of parameters. On the left, we see the architecture for GBAF ozfatura2022all, and on the right, we see the architecture for LightCode (ours).
• Figure 4: Feature extractor design for LightCode for a rate $3/9$ code.
• Figure 5: Noiseless feedback: Performance comparison against existing classical and neural feedback codes for rate $3/9$. PowerBlast achieves the best performance among existing classical schemes and performs comparable to state-of-the-art neural coding schemes in high-SNR regions. LightCode achieves superior BLER performance compared to GBAF while utilizing $< 1/10^{\text{th}}$ the number of parameters.

Theorems & Definitions (3)

• Theorem 1: Error Analysis
• proof
• Remark 1