Hybrid Federated Learning for Noise-Robust Training

TL;DR

Hybrid Federated Learning (HFL) addresses the noise sensitivity of FL and the potential accuracy gap of FD by allowing each UE to transmit either gradients or logits in each round, with a global convex combination parameter $\alpha$ guiding the update. The paper provides a convergence analysis under standard strong-convexity and bounded-variance assumptions and introduces two DoF mechanisms: adaptive UE clustering via Jenks optimization of the post-equalization noise factor and adaptive weight selection via a damped Newton method. Empirical results at low SNR show HFL achieving higher test accuracy and faster convergence than FL or FD, validating the noise-robustness and learning-speed trade-off. The work advances practical, privacy-preserving distributed learning over wireless networks and suggests avenues for beamforming and more DoFs.

Abstract

Federated learning (FL) and federated distillation (FD) are distributed learning paradigms that train UE models with enhanced privacy, each offering different trade-offs between noise robustness and learning speed. To mitigate their respective weaknesses, we propose a hybrid federated learning (HFL) framework in which each user equipment (UE) transmits either gradients or logits, and the base station (BS) selects the per-round weights of FL and FD updates. We derive convergence of HFL framework and introduce two methods to exploit degrees of freedom (DoF) in HFL, which are (i) adaptive UE clustering via Jenks optimization and (ii) adaptive weight selection via a damped Newton method. Numerical results show that HFL achieves superior test accuracy at low SNR when both DoF are exploited.

Figures (3)

• Figure 1: HFL framework at round $t$.
• Figure 2: Comparison of HFL, FL, and FD at low SNR.
• Figure 3: Effect of DoF at low SNR: Clustering and ratio selection.

Theorems & Definitions (2)

• Proposition 3.1
• proof