Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Hierarchical Federated Learning Approach for the Internet of Things

Seyed Mohammad Azimi-Abarghouyi, Viktoria Fodor

TL;DR

This work addresses scalable federated learning for large-scale IoT deployments by introducing QHetFed, a two-level hierarchical FL algorithm that combines intra-set gradient aggregation with inter-set model aggregation and uses end-of-interval gamma steps to accelerate learning under data heterogeneity and quantization. It provides a heterogeneity-aware convergence analysis that yields an optimality gap bound dependent on quantization noise and data heterogeneity, and derives closed-form expressions for optimal intra-set iterations and gradient steps under a deadline. System optimization links communication and computation times to learning parameters, enabling closed-form selection of tau and gamma to minimize gradient error within time constraints. Empirical results on CIFAR-10 with a three-set IoT-like topology show QHetFed is more robust to non-iid data and quantization than conventional hierarchical schemes, with optimized parameter choices delivering faster convergence and higher final accuracy in heterogeneous settings.

Abstract

This paper presents a novel federated learning solution, QHetFed, suitable for large-scale Internet of Things deployments, addressing the challenges of large geographic span, communication resource limitation, and data heterogeneity. QHetFed is based on hierarchical federated learning over multiple device sets, where the learning process and learning parameters take the necessary data quantization and the data heterogeneity into consideration to achieve high accuracy and fast convergence. Unlike conventional hierarchical federated learning algorithms, the proposed approach combines gradient aggregation in intra-set iterations with model aggregation in inter-set iterations. We offer a comprehensive analytical framework to evaluate its optimality gap and convergence rate, and give a closed form expression for the optimal learning parameters under a deadline, that accounts for communication and computation times. Our findings reveal that QHetFed consistently achieves high learning accuracy and significantly outperforms other hierarchical algorithms, particularly in scenarios with heterogeneous data distributions.

A Hierarchical Federated Learning Approach for the Internet of Things

TL;DR

This work addresses scalable federated learning for large-scale IoT deployments by introducing QHetFed, a two-level hierarchical FL algorithm that combines intra-set gradient aggregation with inter-set model aggregation and uses end-of-interval gamma steps to accelerate learning under data heterogeneity and quantization. It provides a heterogeneity-aware convergence analysis that yields an optimality gap bound dependent on quantization noise and data heterogeneity, and derives closed-form expressions for optimal intra-set iterations and gradient steps under a deadline. System optimization links communication and computation times to learning parameters, enabling closed-form selection of tau and gamma to minimize gradient error within time constraints. Empirical results on CIFAR-10 with a three-set IoT-like topology show QHetFed is more robust to non-iid data and quantization than conventional hierarchical schemes, with optimized parameter choices delivering faster convergence and higher final accuracy in heterogeneous settings.

Abstract

This paper presents a novel federated learning solution, QHetFed, suitable for large-scale Internet of Things deployments, addressing the challenges of large geographic span, communication resource limitation, and data heterogeneity. QHetFed is based on hierarchical federated learning over multiple device sets, where the learning process and learning parameters take the necessary data quantization and the data heterogeneity into consideration to achieve high accuracy and fast convergence. Unlike conventional hierarchical federated learning algorithms, the proposed approach combines gradient aggregation in intra-set iterations with model aggregation in inter-set iterations. We offer a comprehensive analytical framework to evaluate its optimality gap and convergence rate, and give a closed form expression for the optimal learning parameters under a deadline, that accounts for communication and computation times. Our findings reveal that QHetFed consistently achieves high learning accuracy and significantly outperforms other hierarchical algorithms, particularly in scenarios with heterogeneous data distributions.
Paper Structure (10 sections, 6 theorems, 92 equations, 9 figures, 3 tables, 2 algorithms)

This paper contains 10 sections, 6 theorems, 92 equations, 9 figures, 3 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. State of the Art
  3. Contributions
  4. Proposed Hierarchical Scheme
  5. Learning Algorithm
  6. Convergence Analysis
  7. Comparison with the Conventional Hierarchical Scheme
  8. System Optimization
  9. Experimental Results
  10. Conclusions

Key Result

Theorem 1

Under the following conditions on the learning rate $\mu$: and the optimality gap of QHetFed is characterized as where

Figures (9)

  • Figure 1: Hierarchical system. As an example of data heterogeneity, three different environments--- shopping mall, library, and stadium--- are illustrated for sets 1, 2, and $C$, respectively.
  • Figure 2: Test accuracy as a function of global iterations (i.i.d.)
  • Figure 3: Test accuracy as a function of global iterations (i.i.d.)
  • Figure 4: Test accuracy as a function of global iterations (non-i.i.d.)
  • Figure 5: Test accuracy as a function of run-time (i.i.d.)
  • ...and 4 more figures

Theorems & Definitions (14)

  • Definition 1
  • Theorem 1
  • Remark 1
  • Remark 2
  • Remark 3
  • Corollary 1
  • Remark 4
  • Corollary 2
  • Corollary 3
  • Lemma 1
  • ...and 4 more