Learning Binary Autoencoder-Based Codes with Progressive Training
Vukan Ninkovic, Dejan Vukobratovic
TL;DR
This paper tackles learning binary error-correcting codes with autoencoders by addressing gradient-flow challenges associated with discretization. It introduces a two-stage training pipeline—continuous pretraining followed by direct binarization—that yields a discrete codebook while preserving end-to-end optimization. On the $n=7$, $k=4$ block with a binary symmetric channel, the method discovers a coset of the Hamming$ (7,4)$ code, achieving BLER identical to ML decoding and revealing a linear code with the same distance spectrum. The results demonstrate that compact neural architectures can implicitly learn algebraically optimal binary codes, offering a practical route toward end-to-end learning of structured codes, with future work targeting longer blocks and scalable message representations.
Abstract
Error correcting codes play a central role in digital communication, ensuring that transmitted information can be accurately reconstructed despite channel impairments. Recently, autoencoder (AE) based approaches have gained attention for the end-to-end design of communication systems, offering a data driven alternative to conventional coding schemes. However, enforcing binary codewords within differentiable AE architectures remains difficult, as discretization breaks gradient flow and often leads to unstable convergence. To overcome this limitation, a simplified two stage training procedure is proposed, consisting of a continuous pretraining phase followed by direct binarization and fine tuning without gradient approximation techniques. For the (7,4) block configuration over a binary symmetric channel (BSC), the learned encoder-decoder pair learns a rotated version (coset code) of the optimal Hamming code, naturally recovering its linear and distance properties and thereby achieving the same block error rate (BLER) with maximum likelihood (ML) decoding. These results indicate that compact AE architectures can effectively learn structured, algebraically optimal binary codes through stable and straightforward training.