Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hierarchical Over-the-Air Federated Learning with Awareness of Interference and Data Heterogeneity

Seyed Mohammad Azimi-Abarghouyi, Viktoria Fodor

TL;DR

This work tackles scalable hierarchical federated learning over wireless networks by addressing interference and data heterogeneity with a novel method, MultiAirFed, and a scalable over-the-air transmission scheme. It couples intra-cluster gradient aggregations with inter-cluster model-parameter aggregations across a PCP-based multi-cluster topology, using gradient transmissions to improve robustness to noise and heterogeneity. The authors derive mean squared error distortions for uplink and downlink aggregations, and provide optimal normalizing factors to minimize distortion, demonstrating that their approach outperforms conventional hierarchical FL, particularly under strong interference and non-i.i.d. data. The practical impact lies in achieving high learning accuracy with minimal resource usage through OTA aggregation, making large-scale hierarchical FL more feasible in real wireless deployments.

Abstract

When implementing hierarchical federated learning over wireless networks, scalability assurance and the ability to handle both interference and device data heterogeneity are crucial. This work introduces a learning method designed to address these challenges, along with a scalable transmission scheme that efficiently uses a single wireless resource through over-the-air computation. To provide resistance against data heterogeneity, we employ gradient aggregations. Meanwhile, the impact of interference is minimized through optimized receiver normalizing factors. For this, we model a multi-cluster wireless network using stochastic geometry, and characterize the mean squared error of the aggregation estimations as a function of the network parameters. We show that despite the interference and the data heterogeneity, the proposed scheme achieves high learning accuracy and can significantly outperform the conventional hierarchical algorithm.

Hierarchical Over-the-Air Federated Learning with Awareness of Interference and Data Heterogeneity

TL;DR

This work tackles scalable hierarchical federated learning over wireless networks by addressing interference and data heterogeneity with a novel method, MultiAirFed, and a scalable over-the-air transmission scheme. It couples intra-cluster gradient aggregations with inter-cluster model-parameter aggregations across a PCP-based multi-cluster topology, using gradient transmissions to improve robustness to noise and heterogeneity. The authors derive mean squared error distortions for uplink and downlink aggregations, and provide optimal normalizing factors to minimize distortion, demonstrating that their approach outperforms conventional hierarchical FL, particularly under strong interference and non-i.i.d. data. The practical impact lies in achieving high learning accuracy with minimal resource usage through OTA aggregation, making large-scale hierarchical FL more feasible in real wireless deployments.

Abstract

When implementing hierarchical federated learning over wireless networks, scalability assurance and the ability to handle both interference and device data heterogeneity are crucial. This work introduces a learning method designed to address these challenges, along with a scalable transmission scheme that efficiently uses a single wireless resource through over-the-air computation. To provide resistance against data heterogeneity, we employ gradient aggregations. Meanwhile, the impact of interference is minimized through optimized receiver normalizing factors. For this, we model a multi-cluster wireless network using stochastic geometry, and characterize the mean squared error of the aggregation estimations as a function of the network parameters. We show that despite the interference and the data heterogeneity, the proposed scheme achieves high learning accuracy and can significantly outperform the conventional hierarchical algorithm.
Paper Structure (7 sections, 2 theorems, 28 equations, 2 figures, 1 table)

This paper contains 7 sections, 2 theorems, 28 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. System Model
  3. Proposed Learning Method
  4. Transmission Scheme
  5. Experimental Results
  6. Conclusions
  7. Proof of Theorem 1

Key Result

Theorem 1

The inter-cluster uplink interference power is

Figures (2)

  • Figure 1: A hierarchical FL network, where three clusters with two learning tasks are illustrated, and MultiAirFed method.
  • Figure 2: Test accuracy (a) for different $C$ values, (b) for non-i.i.d. data and $\lambda_\text{p} = 20 \ \text{Km}^{-2}$, (c) for i.i.d. data and $\lambda_\text{p} = 40 \ \text{Km}^{-2}$.

Theorems & Definitions (4)

  • Theorem 1
  • Theorem 2
  • Remark 1
  • Remark 2