Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedADP: Unified Model Aggregation for Federated Learning with Heterogeneous Model Architectures

Jiacheng Wang, Hongtao Lv, Lei Liu

TL;DR

This work tackles federated learning under heterogeneous client architectures and resource constraints. It introduces FedADP, a framework that aligns diverse client models to a unified structure for aggregation using a NetChange-based mechanism, then redistributes updated models back to clients in their native configurations. Key components include To-Wider/To-Deeper and To-Narrower/To-Shallower operations to adapt architectures while preserving performance, enabling full participation of heterogeneous clients. Experiments on MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 with VGG variants show FedADP outperforms FlexiFed, Clustered-FL, and Standalone, with substantial accuracy gains and improved convergence in highly heterogeneous settings. The approach promises practical impact for scalable, efficient, and accurate FL in real-world deployments with diverse hardware capabilities.

Abstract

Traditional Federated Learning (FL) faces significant challenges in terms of efficiency and accuracy, particularly in heterogeneous environments where clients employ diverse model architectures and have varying computational resources. Such heterogeneity complicates the aggregation process, leading to performance bottlenecks and reduced model generalizability. To address these issues, we propose FedADP, a federated learning framework designed to adapt to client heterogeneity by dynamically adjusting model architectures during aggregation. FedADP enables effective collaboration among clients with differing capabilities, maximizing resource utilization and ensuring model quality. Our experimental results demonstrate that FedADP significantly outperforms existing methods, such as FlexiFed, achieving an accuracy improvement of up to 23.30%, thereby enhancing model adaptability and training efficiency in heterogeneous real-world settings.

FedADP: Unified Model Aggregation for Federated Learning with Heterogeneous Model Architectures

TL;DR

This work tackles federated learning under heterogeneous client architectures and resource constraints. It introduces FedADP, a framework that aligns diverse client models to a unified structure for aggregation using a NetChange-based mechanism, then redistributes updated models back to clients in their native configurations. Key components include To-Wider/To-Deeper and To-Narrower/To-Shallower operations to adapt architectures while preserving performance, enabling full participation of heterogeneous clients. Experiments on MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 with VGG variants show FedADP outperforms FlexiFed, Clustered-FL, and Standalone, with substantial accuracy gains and improved convergence in highly heterogeneous settings. The approach promises practical impact for scalable, efficient, and accurate FL in real-world deployments with diverse hardware capabilities.

Abstract

Traditional Federated Learning (FL) faces significant challenges in terms of efficiency and accuracy, particularly in heterogeneous environments where clients employ diverse model architectures and have varying computational resources. Such heterogeneity complicates the aggregation process, leading to performance bottlenecks and reduced model generalizability. To address these issues, we propose FedADP, a federated learning framework designed to adapt to client heterogeneity by dynamically adjusting model architectures during aggregation. FedADP enables effective collaboration among clients with differing capabilities, maximizing resource utilization and ensuring model quality. Our experimental results demonstrate that FedADP significantly outperforms existing methods, such as FlexiFed, achieving an accuracy improvement of up to 23.30%, thereby enhancing model adaptability and training efficiency in heterogeneous real-world settings.
Paper Structure (18 sections, 3 equations, 4 figures, 1 table, 3 algorithms)

This paper contains 18 sections, 3 equations, 4 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Personalized Federated Learning
  4. Heterogeneous Network Structure
  5. FedADP
  6. FedADP Framework
  7. NetChange
  8. To-Wider and To-Deeper
  9. To-Narrower and To-Shallower
  10. EVALUATION
  11. Experimental Setup
  12. Datasets
  13. Training Model
  14. Baselines
  15. Other settings
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Some variations of the VGG model.
  • Figure 2: Difference between traditional PFL and FedADP in the workflow
  • Figure 3: Other forms of the VGG model: VGG-14, VGG-15, VGG-17, and VGG-18.
  • Figure 4: The experimental results compare the performance of FedADP, FlexiFed, Cluster-FL, and Standalone on the MNIST, F-MNIST, CIFAR-10, and CIFAR-100 datasets.