Exploiting Label-Aware Channel Scoring for Adaptive Channel Pruning in Split Learning

TL;DR

Experimental results show that ACP-SL consistently outperforms benchmark schemes in test accuracy, and it reaches a target test accuracy in fewer training rounds, thereby reducing communication overhead.

Abstract

Split learning (SL) transfers most of the training workload to the server, which alleviates computational burden on client devices. However, the transmission of intermediate feature representations, referred to as smashed data, incurs significant communication overhead, particularly when a large number of client devices are involved. To address this challenge, we propose an adaptive channel pruning-aided SL (ACP-SL) scheme. In ACP-SL, a label-aware channel importance scoring (LCIS) module is designed to generate channel importance scores, distinguishing important channels from less important ones. Based on these scores, an adaptive channel pruning (ACP) module is developed to prune less important channels, thereby compressing the corresponding smashed data and reducing the communication overhead. Experimental results show that ACP-SL consistently outperforms benchmark schemes in test accuracy. Furthermore, it reaches a target test accuracy in fewer training rounds, thereby reducing communication overhead.

Figures (6)

• Figure 1: Block diagram of the proposed ACP-SL scheme.
• Figure 2: Effect of channel importance on the training loss of SL.
• Figure 3: Effects of the instantaneous channel importance score $S_{i,\rm{Inst}}^t$, the historical channel importance score ${S}_{i,\rm{Hist}}^t$, and the combined channel importance score ${S}_{i,\rm{Comb}}^t$ on the training loss and test accuracy of SL.
• Figure 4: Test accuracy on CIFAR-10 and Fashion-MNIST under IID and non-IID settings.
• Figure 5: Ablation experiment results of the proposed LCIS on CIFAR-10 and Fashion-MNIST under the non-IID setting.