MinGRU-Based Encoder for Turbo Autoencoder Frameworks

TL;DR

This work tackles scalability in neural channel coding for long sequences by integrating an efficient minGRU-based encoder with a Mamba block into a parallel Turbo autoencoder. The approach preserves competitive BLER/BER performance relative to CNN-based TurboAE while delivering superior training efficiency and scalability for longer block lengths, leveraging parallelizable RNN components. By combining sequential modeling with parallel processing in an SS(M)-based framework, the paper demonstrates a practical path toward scalable end-to-end neural channel codes. Future directions include applying the design to the decoder, optimizing for very long sequences, and pursuing cuDNN-accelerated implementations to maximize practical throughput.

Abstract

Early neural channel coding approaches leveraged dense neural networks with one-hot encodings to design adaptive encoder-decoder pairs, improving block error rate (BLER) and automating the design process. However, these methods struggled with scalability as the size of message sets and block lengths increased. TurboAE addressed this challenge by focusing on bit-sequence inputs rather than symbol-level representations, transforming the scalability issue associated with large message sets into a sequence modeling problem. While recurrent neural networks (RNNs) were a natural fit for sequence processing, their reliance on sequential computations made them computationally expensive and inefficient for long sequences. As a result, TurboAE adopted convolutional network blocks, which were faster to train and more scalable, but lacked the sequential modeling advantages of RNNs. Recent advances in efficient RNN architectures, such as minGRU and minLSTM, and structured state space models (SSMs) like S4 and S6, overcome these limitations by significantly reducing memory and computational overhead. These models enable scalable sequence processing, making RNNs competitive for long-sequence tasks. In this work, we revisit RNNs for Turbo autoencoders by integrating the lightweight minGRU model with a Mamba block from SSMs into a parallel Turbo autoencoder framework. Our results demonstrate that this hybrid design matches the performance of convolutional network-based Turbo autoencoder approaches for short sequences while significantly improving scalability and training efficiency for long block lengths. This highlights the potential of efficient RNNs in advancing neural channel coding for long-sequence scenarios.

Figures (6)

• Figure 1: Mamba block with minGRU module. The activations are chosen to be Swish/SILU activations. The projection/transformation upscale the input to the hidden size, which is a parameter and is mostly chosen to be $<12$ here. The multiplication is a multiplicative gate which can be thought as an adaptively learned element-wise filter operation.
• Figure 2: Experiments for various feature sizes for the minGRU encoder blocks. Showcasing the behavior of the BER for encoder training over $500$ epochs.
• Figure 3: Experiments for various feature sizes for the minGRU encoder blocks. Showcasing the behavior of the training loss for encoder training over $500$ epochs.
• Figure 4: Training dynamics for the tested models.
• Figure 5: Bler values for various models. All models were tested for $N=50000$ samples per $E_b/N_0$ point until $E_b/N_0=3.5$ dB, and $2N$ samples from $4$ dB, $3N$ from $5$ dB and $4N$ from $5.5$ dB.