FlexFL: Heterogeneous Federated Learning via APoZ-Guided Flexible Pruning in Uncertain Scenarios

TL;DR

FlexFL tackles heterogeneity and uncertain resource constraints in Federated Learning for AIoT by combining APoZ-guided flexible pruning with adaptive local pruning and self-knowledge distillation. A proxy-data–driven APoZScore-based model generation creates a model pool of heterogeneous submodels, while AdaPrune adjusts models to device resources and self-KD propagates knowledge from small to large models during local training. Empirical results on CIFAR-10/100 and Tiny ImageNet, plus real-world FEMNIST and LEAF Widar datasets, show FlexFL achieving up to 14.24% improvement for large models and superior average-model performance with reduced communication overhead. The approach demonstrates robustness to device heterogeneity, data non-IIDness, and real-world AIoT deployments, offering practical gains for scalable, privacy-preserving collaborative learning.

Abstract

Along with the increasing popularity of Deep Learning (DL) techniques, more and more Artificial Intelligence of Things (AIoT) systems are adopting federated learning (FL) to enable privacy-aware collaborative learning among AIoT devices. However, due to the inherent data and device heterogeneity issues, existing FL-based AIoT systems suffer from the model selection problem. Although various heterogeneous FL methods have been investigated to enable collaborative training among heterogeneous models, there is still a lack of i) wise heterogeneous model generation methods for devices, ii) consideration of uncertain factors, and iii) performance guarantee for large models, thus strongly limiting the overall FL performance. To address the above issues, this paper introduces a novel heterogeneous FL framework named FlexFL. By adopting our Average Percentage of Zeros (APoZ)-guided flexible pruning strategy, FlexFL can effectively derive best-fit models for heterogeneous devices to explore their greatest potential. Meanwhile, our proposed adaptive local pruning strategy allows AIoT devices to prune their received models according to their varying resources within uncertain scenarios. Moreover, based on self-knowledge distillation, FlexFL can enhance the inference performance of large models by learning knowledge from small models. Comprehensive experimental results show that, compared to state-of-the-art heterogeneous FL methods, FlexFL can significantly improve the overall inference accuracy by up to 14.24%.

Figures (10)

• Figure 1: Framework and workflow of FlexFL.
• Figure 2: Comparison of submodels (VGG16 on CIFAR10).
• Figure 3: Learning curves of FlexFL and three baselines.
• Figure 4: Model pruning ratios similarity with different proxy dataset sizes.
• Figure 5: Learning curves for different numbers of involved devices.