Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Model Compression for Hierarchical Federated Learning

Xi Zhu, Songcan Yu, Junbo Wang, Qinglin Yang

TL;DR

The paper addresses high communication costs and privacy concerns in large-model FL by proposing a hierarchical FL framework that combines adaptive DBSCAN-based clustering with core-client LC aggregation and a two-stage model compression pipeline. A cluster-wide compression matrix and layer-wise sparsification reduce transmission sizes, while LC aggregation minimizes decompression delay; an efficient cluster-wise decompression method guarantees accurate global recovery, supported by Theorem 1. The approach yields comparable accuracy to baselines while significantly reducing energy consumption, enabling scalable, green FL on edge devices. These methods hold practical impact for deploying privacy-preserving learning in bandwidth-constrained, energy-limited environments, such as mobile and IoT networks.

Abstract

Federated learning (FL), as an emerging collaborative learning paradigm, has garnered significant attention due to its capacity to preserve privacy within distributed learning systems. In these systems, clients collaboratively train a unified neural network model using their local datasets and share model parameters rather than raw data, enhancing privacy. Predominantly, FL systems are designed for mobile and edge computing environments where training typically occurs over wireless networks. Consequently, as model sizes increase, the conventional FL frameworks increasingly consume substantial communication resources. To address this challenge and improve communication efficiency, this paper introduces a novel hierarchical FL framework that integrates the benefits of clustered FL and model compression. We present an adaptive clustering algorithm that identifies a core client and dynamically organizes clients into clusters. Furthermore, to enhance transmission efficiency, each core client implements a local aggregation with compression (LC aggregation) algorithm after collecting compressed models from other clients within the same cluster. Simulation results affirm that our proposed algorithms not only maintain comparable predictive accuracy but also significantly reduce energy consumption relative to existing FL mechanisms.

Efficient Model Compression for Hierarchical Federated Learning

TL;DR

The paper addresses high communication costs and privacy concerns in large-model FL by proposing a hierarchical FL framework that combines adaptive DBSCAN-based clustering with core-client LC aggregation and a two-stage model compression pipeline. A cluster-wide compression matrix and layer-wise sparsification reduce transmission sizes, while LC aggregation minimizes decompression delay; an efficient cluster-wise decompression method guarantees accurate global recovery, supported by Theorem 1. The approach yields comparable accuracy to baselines while significantly reducing energy consumption, enabling scalable, green FL on edge devices. These methods hold practical impact for deploying privacy-preserving learning in bandwidth-constrained, energy-limited environments, such as mobile and IoT networks.

Abstract

Federated learning (FL), as an emerging collaborative learning paradigm, has garnered significant attention due to its capacity to preserve privacy within distributed learning systems. In these systems, clients collaboratively train a unified neural network model using their local datasets and share model parameters rather than raw data, enhancing privacy. Predominantly, FL systems are designed for mobile and edge computing environments where training typically occurs over wireless networks. Consequently, as model sizes increase, the conventional FL frameworks increasingly consume substantial communication resources. To address this challenge and improve communication efficiency, this paper introduces a novel hierarchical FL framework that integrates the benefits of clustered FL and model compression. We present an adaptive clustering algorithm that identifies a core client and dynamically organizes clients into clusters. Furthermore, to enhance transmission efficiency, each core client implements a local aggregation with compression (LC aggregation) algorithm after collecting compressed models from other clients within the same cluster. Simulation results affirm that our proposed algorithms not only maintain comparable predictive accuracy but also significantly reduce energy consumption relative to existing FL mechanisms.
Paper Structure (15 sections, 24 equations, 5 figures)

This paper contains 15 sections, 24 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Related Work
  3. Overview of Federated Learning
  4. Green Communication in Federated Learning
  5. System Model
  6. Hierarchical Federated Learning Framework
  7. Problem Definition
  8. Efficient Model Compression based Hierarchical Federated Learning
  9. Adaptive Client Clustering
  10. Model Compression and LC aggregation
  11. Efficient Model Decompression and Decryption
  12. Performance Evaluation
  13. Experimental Setting
  14. Experimental results
  15. Conclusions

Figures (5)

  • Figure 1: Model compression based hierarchical FL
  • Figure 2: The Architecture of Efficient Model Compression based Hierarchical Federated Learning Framework
  • Figure 3: Accuracy with different method on two datasets
  • Figure 4: Impact of dataset & core-client selection on cost
  • Figure 5: Influence of radius of neighborhood on energy cost