FedX: Explanation-Guided Pruning for Communication-Efficient Federated Learning in Remote Sensing

TL;DR

Experimental results show the success of FedX in significantly reducing the number of shared model parameters while enhancing the generalization capability of the global model, compared to both unpruned model and state-of-the-art pruning methods.

Abstract

Federated learning (FL) enables the collaborative training of deep neural networks across decentralized data archives (i.e., clients), where each client stores data locally and only shares model updates with a central server. This makes FL a suitable learning paradigm for remote sensing (RS) image classification tasks, where data centralization may be restricted due to legal and privacy constraints. However, a key challenge in applying FL to RS tasks is the communication overhead caused by the frequent exchange of large model updates between clients and the central server. To address this issue, in this paper we propose a novel strategy (denoted as FedX) that uses explanation-guided pruning to reduce communication overhead by minimizing the size of the transmitted models without compromising performance. FedX leverages backpropagation-based explanation methods to estimate the task-specific importance of model components and prunes the least relevant ones at the central server. The resulting sparse global model is then sent to clients, substantially reducing communication overhead. We evaluate FedX on multi-label scene classification using the BigEarthNet-S2 dataset and single-label scene classification using the EuroSAT dataset. Experimental results show the success of FedX in significantly reducing the number of shared model parameters while enhancing the generalization capability of the global model, compared to both unpruned model and state-of-the-art pruning methods. The code of FedX will be available at https://git.tu-berlin.de/rsim/FedX.

Figures (5)

• Figure 1: Overview of FedX. (1) Clients perform local training on their private RS data. (2) The central server aggregates the locally trained models. (3) Reference RS data is used to compute explanation-based relevance scores, from which a global mask matrix is derived. (4) The global model is pruned using the mask matrix, resulting in a sparsified model distributed to clients for the next training round.
• Figure 2: mAP scores obtained by FedX with different pruning configurations (layer-wise and global), relevance-based explanation methods (LRP and IG), and a magnitude-based pruning method for ResNet18 under varying pruning rates (the BigEarthNet-S2 dataset).
• Figure 3: Distribution of retained and pruned parameters per layer in ResNet18 under 90% overall sparsity, comparing global and layer-wise pruning configurations for both datasets. Yellow segments indicate retained convolutional parameters, red segments indicate pruned parameters, and the blue bar represents the fully connected layer (which is not subject to pruning).
• Figure 4: Result scores obtained by FedX (using the LRP method with a layer-wise configuration) for different model architectures under varying pruning rates for: (a) the BigEarthNet-S2 (mAP); and (b) the EuroSAT (accuracy) datasets.
• Figure 5: mAP scores obtained by FedX (using the LRP method with a layer-wise configuration), FedDST, FedDIP, PruneFL, LotteryFL, and random pruning for ResNet6 under varying pruning rates (the BigEarthNet-S2 dataset).