Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Noise-Robust and Resource-Efficient ADMM-based Federated Learning

Ehsan Lari, Reza Arablouei, Vinay Chakravarthi Gogineni, Stefan Werner

TL;DR

This work tackles federated learning under noisy communication channels and limited client participation by formulating an ADMM-based WLS solver that is robust to additive noise and reduces communication overhead. It introduces dual-variable elimination and random client scheduling, plus a continual local-update variant, to boost robustness and efficiency. The authors provide rigorous mean and mean-square convergence analyses and derive a closed-form steady-state MSE, validated by simulations that confirm the theory and demonstrate significant performance gains and resource savings. The approach offers a practical path to reliable, scalable FL in wireless environments with heterogeneous devices.

Abstract

Federated learning (FL) leverages client-server communications to train global models on decentralized data. However, communication noise or errors can impair model accuracy. To address this problem, we propose a novel FL algorithm that enhances robustness against communication noise while also reducing communication load. We derive the proposed algorithm through solving the weighted least-squares (WLS) regression problem as an illustrative example. We first frame WLS regression as a distributed convex optimization problem over a federated network employing random scheduling for improved communication efficiency. We then apply the alternating direction method of multipliers (ADMM) to iteratively solve this problem. To counteract the detrimental effects of cumulative communication noise, we introduce a key modification by eliminating the dual variable and implementing a new local model update at each participating client. This subtle yet effective change results in using a single noisy global model update at each client instead of two, improving robustness against additive communication noise. Furthermore, we incorporate another modification enabling clients to continue local updates even when not selected by the server, leading to substantial performance improvements. Our theoretical analysis confirms the convergence of our algorithm in both mean and the mean-square senses, even when the server communicates with a random subset of clients over noisy links at each iteration. Numerical results validate the effectiveness of our proposed algorithm and corroborate our theoretical findings.

Noise-Robust and Resource-Efficient ADMM-based Federated Learning

TL;DR

This work tackles federated learning under noisy communication channels and limited client participation by formulating an ADMM-based WLS solver that is robust to additive noise and reduces communication overhead. It introduces dual-variable elimination and random client scheduling, plus a continual local-update variant, to boost robustness and efficiency. The authors provide rigorous mean and mean-square convergence analyses and derive a closed-form steady-state MSE, validated by simulations that confirm the theory and demonstrate significant performance gains and resource savings. The approach offers a practical path to reliable, scalable FL in wireless environments with heterogeneous devices.

Abstract

Federated learning (FL) leverages client-server communications to train global models on decentralized data. However, communication noise or errors can impair model accuracy. To address this problem, we propose a novel FL algorithm that enhances robustness against communication noise while also reducing communication load. We derive the proposed algorithm through solving the weighted least-squares (WLS) regression problem as an illustrative example. We first frame WLS regression as a distributed convex optimization problem over a federated network employing random scheduling for improved communication efficiency. We then apply the alternating direction method of multipliers (ADMM) to iteratively solve this problem. To counteract the detrimental effects of cumulative communication noise, we introduce a key modification by eliminating the dual variable and implementing a new local model update at each participating client. This subtle yet effective change results in using a single noisy global model update at each client instead of two, improving robustness against additive communication noise. Furthermore, we incorporate another modification enabling clients to continue local updates even when not selected by the server, leading to substantial performance improvements. Our theoretical analysis confirms the convergence of our algorithm in both mean and the mean-square senses, even when the server communicates with a random subset of clients over noisy links at each iteration. Numerical results validate the effectiveness of our proposed algorithm and corroborate our theoretical findings.
Paper Structure (21 sections, 60 equations, 10 figures, 2 algorithms)

This paper contains 21 sections, 60 equations, 10 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Federated Learning over Noisy Channels
  3. Federated Weighted Least-Squares Regression
  4. Dual Variable Elimination
  5. Communication Noise
  6. Resource-efficient FL over Noisy Channels
  7. Random Scheduling
  8. RERCE-Fed
  9. RERCE-Fed with Continual Local Updates
  10. Performance Analysis
  11. Mean Convergence
  12. Mean-Square Convergence
  13. Steady-State Mean-Square Error
  14. Simulation Results
  15. Performance of RERCE-Fed
  16. ...and 6 more sections

Figures (10)

  • Figure 1: NMSE of \ref{['eq:FD']}-\ref{['eq:FDxbar1']} and \ref{['fl3']} for ${\mathcal{C}} = K = 100$.
  • Figure 2: NMSE of \ref{['eq:FD']}-\ref{['eq:FDxbar1']} with ${\mathcal{C}} = 4$ and \ref{['fl3']} with ${\mathcal{C}} \in \{4,75,90\}$.
  • Figure 3: NMSE of \ref{['fl3']} with ${\mathcal{C}} = 4$ and RERCE-Fed \ref{['eq:rercefed']} for different numbers of participating clients ${\mathcal{C}} \in \{4,10,25\}$.
  • Figure 4: NMSE of RERCE-Fed \ref{['eq:rercefed']} and RERCE-Fed with continual local updates \ref{['eq:clu']} for ${\mathcal{C}} = 4$ and different uplink and downlink noise variances $\sigma^2_{\eta_k} = \sigma^2_{\zeta_k} \in \{ 6.25 \times 10^{-4},10^{-2} \}$.
  • Figure 5: NMSE of RERCE-Fed \ref{['eq:rercefed']} and RERCE-Fed with continual local updates \ref{['eq:clu']} for ${\mathcal{C}} = 10$ and different uplink and downlink noise variances $\sigma^2_{\eta_k} = \sigma^2_{\zeta_k} \in \{ 6.25 \times 10^{-4},10^{-2} \}$.
  • ...and 5 more figures