Higher-order Interpretations of Deepcode, a Learned Feedback Code

TL;DR

The paper enriches the interpretability of Deepcode by introducing an enc 3, dec 4 two-stage model that enables higher-order error correction through multi-step feedback and a bidirectional decoder. It defines first-, second-, and third-order correction mechanisms, including entanglement across time steps, and supports these with an error-analysis workflow (outlier analysis and clustering) to identify patterns causing decoding failures. Empirical results show improved BER at low forward SNR and competitive performance against Deepcode with many hidden states, while drastically reducing parameter count to ~90 (vs tens of thousands). The approach demonstrates that combining interpretable analytical structure with learned parameters can yield robust, efficient learned-feedback codes suitable for practical noisy-feedback channels.

Abstract

We present an interpretation of Deepcode, a learned feedback code that showcases higher-order error correction relative to an earlier interpretable model. By interpretation, we mean succinct analytical encoder and decoder expressions (albeit with learned parameters) in which the role of feedback in achieving error correction is easy to understand. By higher-order, we mean that longer sequences of large noise values are acted upon by the encoder (which has access to these through the feedback) and used in error correction at the decoder in a two-stage decoding process.

Figures (9)

• Figure 1: Learned parameters $\theta$ and $\phi$ in the encoding $f_\theta$ and decoding $g_\theta$ functions for transmission over the AWGN channel with passive noisy feedback. Forward noise is i.i.d. Gaussian ${\cal N}(0,\sigma_f^2)$ while feedback noise is also i.i.d. Gaussian ${\cal N}(0,\sigma_{fb}^2)$.
• Figure 2: Deepcode encoder (above) and decoder (below). Here, $i \in \{1, \ldots, K + 1\}$ because of zero padding. We omit power allocation and normalization for simplicity.
• Figure 3: Clustering error features leading to decoding errors in the (enc 2, dec 2) single-stage interpretable model corrected by Deepcode with $7$ hidden states.
• Figure 4: An example of extended noise events occurring for $b_i=0$. The solid line represents noise events that will cause outliers and alter the future parity bits for error correction, whereas the dashed line indicates that no further error correction is needed. We omit the coefficients in the decoding for simplicity.
• Figure 5: Effect of problematic bit $b_i$ on the previous bit

Theorems & Definitions (1)

• Definition 1