NSC-SL: A Bandwidth-Aware Neural Subspace Compression for Communication-Efficient Split Learning

TL;DR

NSC-SL tackles the communication bottleneck in distributed split learning by introducing bandwidth-aware adaptive rank selection (BAS) and orthogonal alternating subspace approximation with an error-correction loop (OASA+ECL). BAS dynamically selects the compression rank to satisfy energy preservation and bandwidth constraints, using a randomized spectral estimate and a budget-aware cap, with the final rank computed as $r = \min\{ r_{\eta}, \left\lfloor \frac{B_{\text{max}}}{4(m+n)}\right\rfloor, r_{\text{cap}} \}$. OASA+ECL iteratively refines a low-rank approximation by alternating updates with residual feedback, yielding $\hat{M} = P^{(T)} (Q^{(T)})^{\top}$ and a stabilized convergence with complexity $O(mnr)$. Empirical results on HAM10000 with ResNet-18 show NSC-SL outperforms baselines in both accuracy and MSE under various bandwidths, indicating practical gains for privacy-preserving, communication-efficient edge learning.

Abstract

The expanding scale of neural networks poses a major challenge for distributed machine learning, particularly under limited communication resources. While split learning (SL) alleviates client computational burden by distributing model layers between clients and server, it incurs substantial communication overhead from frequent transmission of intermediate activations and gradients. To tackle this issue, we propose NSC-SL, a bandwidth-aware adaptive compression algorithm for communication-efficient SL. NSC-SL first dynamically determines the optimal rank of low-rank approximation based on the singular value distribution for adapting real-time bandwidth constraints. Then, NSC-SL performs error-compensated tensor factorization using alternating orthogonal iteration with residual feedback, effectively minimizing truncation loss. The collaborative mechanisms enable NSC-SL to achieve high compression ratios while preserving semantic-rich information essential for convergence. Extensive experiments demonstrate the superb performance of NSC-SL.

Figures (3)

• Figure 1: Architecture of the proposed NSC-SL framework.
• Figure 2: The test performance of ResNet-18 on the HAM10000 datasets.
• Figure 3: The ablation experiments for OASA on the HAM10000 dataset without iteration and without ECL.