Spectral-Aligned Pruning for Universal Error-Correcting Code Transformers

TL;DR

The paper tackles the high compute and memory demands of universal transformer-based ECC decoders by introducing Spectral-Aligned Pruning (SAP), a structure-aware framework that reuses pruning masks across spectrally similar codes and applies LoRA-based, per-code recovery on a shared pruned backbone. SAP leverages the spectrum of each code's bipartite graph to guide a nearest-neighbor mask retrieval with a threshold-based reuse decision, expanding the mask library only when needed. Empirically, SAP achieves decoding performance comparable to dedicated per-code pruning while delivering substantial FLOPs reductions and modest per-code parameter overhead through LoRA adapters, demonstrated across BCH, LDPC, polar, and 5G NR LDPC codes. A key finding is the strong correlation (Pearson $\rho \approx 0.94$) between bipartite-graph adjacency-spectrum similarity and pruning-mask overlap, validating spectrum-based retrieval as a principled, scalable approach for cross-code pruning mask reuse. Overall, SAP enables efficient deployment of universal ECC decoders by balancing cross-code mask reuse with code-specific recovery, with potential gains when combined with quantization and further sparsification.

Abstract

Recently, the Foundation Error Correction Code Transformer (FECCT) has emerged as a promising universal channel decoder, achieving competitive decoding performance across diverse code families by relying on a single shared model backbone, optionally followed by code-specific retraining. Despite this flexibility, the high computational complexity and large parameter footprint of transformer-based decoders present substantial obstacles to practical deployment. To address these challenges, we investigate structured pruning for FECCT and propose Spectral-Aligned Pruning (SAP), a structure-aware framework that enables cross-code reuse of structured pruning masks across codes by leveraging the spectrum of the corresponding bipartite graph. After pruning, SAP performs per-code recovery via parameter-efficient low-rank adaptation (LoRA), enabling a shared pruned backbone while storing only small code-specific adapter parameters. Experiments across diverse codes show that SAP achieves decoding performance comparable to dedicated per-code pruning, while enabling substantial reductions in computational cost and model memory footprint through kernel-level structured pruning.

Figures (11)

• Figure 1: Proposed SAP framework.
• Figure 2: Decoded BER comparison of the pretrained FECCT baseline, the dedicated pruned models, and the SAP models.
• Figure 3: Decoded BER on 5G NR LDPC codes. We compare BP with $50$ iterations, dedicated per-code pruning with recovery, and SAP mask reuse with recovery. For both targets, SAP retrieves LDPC $(96,64)$ as the nearest library entry and reuses its structured pruning mask, achieving performance comparable to dedicated pruning.
• Figure 4: Spectral similarity vs. pruning mask similarity (Jaccard index). Pearson correlation: $\rho=0.94$.
• Figure 5: Pruning mask reuse under low spectral similarity ($\kappa = 0.1456$). Reusing a pruning mask derived from a spectrally dissimilar code (LDPC $(49,24)$ and polar $(64,32)$) degrades BER performance relative to SAP's library-selected mask.