FedMHO: Heterogeneous One-Shot Federated Learning Towards Resource-Constrained Edge Devices

TL;DR

This work tackles resource-constrained edge devices in one-shot Federated Learning by introducing FedMHO, a framework that combines deep classification on resource-rich clients with lightweight CVAEs on constrained devices. The server performs data generation and knowledge fusion in two stages, initializes the global model from classification clients, and uses synthetic data to refine it, while strategies FedMHO-MD and FedMHO-SD mitigate knowledge forgetting. An unsupervised data optimization step improves the fidelity of synthetic samples, yielding higher global accuracy than state-of-the-art baselines across four datasets and multiple non-IID partitions. The results demonstrate practical viability for data-free, heterogeneous one-shot FL in edge scenarios, with clear guidance on hyperparameters, sample synthesis, and privacy-utility trade-offs.

Abstract

Federated Learning (FL) is increasingly adopted in edge computing scenarios, where a large number of heterogeneous clients operate under constrained or sufficient resources. The iterative training process in conventional FL introduces significant computation and communication overhead, which is unfriendly for resource-constrained edge devices. One-shot FL has emerged as a promising approach to mitigate communication overhead, and model-heterogeneous FL solves the problem of diverse computing resources across clients. However, existing methods face challenges in effectively managing model-heterogeneous one-shot FL, often leading to unsatisfactory global model performance or reliance on auxiliary datasets. To address these challenges, we propose a novel FL framework named FedMHO, which leverages deep classification models on resource-sufficient clients and lightweight generative models on resource-constrained devices. On the server side, FedMHO involves a two-stage process that includes data generation and knowledge fusion. Furthermore, we introduce FedMHO-MD and FedMHO-SD to mitigate the knowledge-forgetting problem during the knowledge fusion stage, and an unsupervised data optimization solution to improve the quality of synthetic samples. Comprehensive experiments demonstrate the effectiveness of our methods, as they outperform state-of-the-art baselines in various experimental setups.

Figures (9)

• Figure 1: The heterogeneous one-shot Federated Learning (FL) framework. The clients with varying computing capabilities deploy heterogeneous local models. Each client communicates with the central server only once. After fully training their local models on private data, clients send these models to the server, where they are aggregated into a global model.
• Figure 2: Top-1 test accuracy (%) on the EMNIST dataset. 'small' or 'large' refers to the aggregation of only lightweight small local models or only deep large local models, respectively.
• Figure 3: An overview of FedMHO. Resource-sufficient clients train deep classification models, and resource-constrained clients train lightweight generative models. The global model is initialized by averaging local classification models. Synthetic samples are generated to train the global model. To solve the knowledge-forgetting problem, FedMHO-MD employs classification models as teachers to distill the global model, while FedMHO-SD utilizes the initialized global model as a teacher to distill the global model.
• Figure 4: A toy example of the knowledge-forgetting problem.
• Figure 5: Global training epochs to reach target Top-1 test accuracy on the EMNIST dataset.