Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Federated Learning as a Service for Hierarchical Edge Networks with Heterogeneous Models

Wentao Gao, Omid Tavallaie, Shuaijun Chen, Albert Zomaya

TL;DR

This work introduces a communication-efficient model aggregation method designed for FL systems with two-level model aggregations running at the edge and cloud levels that enhances the convergence rate of the global model by leveraging selective knowledge transfer during the aggregation of heterogeneous models.

Abstract

Federated learning (FL) is a distributed Machine Learning (ML) framework that is capable of training a new global model by aggregating clients' locally trained models without sharing users' original data. Federated learning as a service (FLaaS) offers a privacy-preserving approach for training machine learning models on devices with various computational resources. Most proposed FL-based methods train the same model in all client devices regardless of their computational resources. However, in practical Internet of Things (IoT) scenarios, IoT devices with limited computational resources may not be capable of training models that client devices with greater hardware performance hosted. Most of the existing FL frameworks that aim to solve the problem of aggregating heterogeneous models are designed for Independent and Identical Distributed (IID) data, which may make it hard to reach the target algorithm performance when encountering non-IID scenarios. To address these problems in hierarchical networks, in this paper, we propose a heterogeneous aggregation framework for hierarchical edge systems called HAF-Edge. In our proposed framework, we introduce a communication-efficient model aggregation method designed for FL systems with two-level model aggregations running at the edge and cloud levels. This approach enhances the convergence rate of the global model by leveraging selective knowledge transfer during the aggregation of heterogeneous models. To the best of our knowledge, this work is pioneering in addressing the problem of aggregating heterogeneous models within hierarchical FL systems spanning IoT, edge, and cloud environments. We conducted extensive experiments to validate the performance of our proposed method. The evaluation results demonstrate that HAF-Edge significantly outperforms state-of-the-art methods.

Federated Learning as a Service for Hierarchical Edge Networks with Heterogeneous Models

TL;DR

This work introduces a communication-efficient model aggregation method designed for FL systems with two-level model aggregations running at the edge and cloud levels that enhances the convergence rate of the global model by leveraging selective knowledge transfer during the aggregation of heterogeneous models.

Abstract

Federated learning (FL) is a distributed Machine Learning (ML) framework that is capable of training a new global model by aggregating clients' locally trained models without sharing users' original data. Federated learning as a service (FLaaS) offers a privacy-preserving approach for training machine learning models on devices with various computational resources. Most proposed FL-based methods train the same model in all client devices regardless of their computational resources. However, in practical Internet of Things (IoT) scenarios, IoT devices with limited computational resources may not be capable of training models that client devices with greater hardware performance hosted. Most of the existing FL frameworks that aim to solve the problem of aggregating heterogeneous models are designed for Independent and Identical Distributed (IID) data, which may make it hard to reach the target algorithm performance when encountering non-IID scenarios. To address these problems in hierarchical networks, in this paper, we propose a heterogeneous aggregation framework for hierarchical edge systems called HAF-Edge. In our proposed framework, we introduce a communication-efficient model aggregation method designed for FL systems with two-level model aggregations running at the edge and cloud levels. This approach enhances the convergence rate of the global model by leveraging selective knowledge transfer during the aggregation of heterogeneous models. To the best of our knowledge, this work is pioneering in addressing the problem of aggregating heterogeneous models within hierarchical FL systems spanning IoT, edge, and cloud environments. We conducted extensive experiments to validate the performance of our proposed method. The evaluation results demonstrate that HAF-Edge significantly outperforms state-of-the-art methods.
Paper Structure (10 sections, 7 equations, 8 figures, 3 tables, 2 algorithms)

This paper contains 10 sections, 7 equations, 8 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Framework Design
  4. Distance-based Weighting
  5. Edge Aggregation
  6. Cloud Aggregation
  7. Implementation and Evaluation
  8. Experiment Setup
  9. Evaluation Results
  10. Conclusion

Figures (8)

  • Figure 1: Comparison between vanilla FL and Hierarchical FL
  • Figure 2: Entire process of HAF-Edge, where orange and blue arrow represents inner aggregation and communication process, respectively.
  • Figure 3: The changing curves of Euclidean distance between client model and global model per communication round on three clients with varied data distribution.
  • Figure 4: Evaluating HAF-Edge, FedAvg, and MaxCommon strategy for Scenario 1 (2 edge servers, 6 clients for each edge server) using MNIST dataset.
  • Figure 5: Evaluating HAF-Edge, FedAvg, and MaxCommon strategy for Scenario 1 (2 edge servers, 6 clients for each edge server) using FMNIST dataset.
  • ...and 3 more figures