Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

An Element-Wise Weights Aggregation Method for Federated Learning

Yi Hu, Hanchi Ren, Chen Hu, Jingjing Deng, Xianghua Xie

TL;DR

EWWA-FL advances federated learning by introducing element-wise aggregation, where each parameter across local models receives a distinct, client-specific weight rather than a single global proportion. Building on FedAvg, it uses Adam-like first- and second-moment estimates to compute per-parameter contributions and applies Softmax to obtain parameter-level aggregation weights, enabling finer alignment to heterogeneous client data. Empirical results show improved accuracy and faster convergence, particularly on large-class datasets and non-IID settings, with high stability and reduced variance compared to peers like FedOpt and FedCAMS. The approach is implemented in an open-source framework, indicating practical applicability and reproducibility for diverse backbone networks and benchmarks.

Abstract

Federated learning (FL) is a powerful Machine Learning (ML) paradigm that enables distributed clients to collaboratively learn a shared global model while keeping the data on the original device, thereby preserving privacy. A central challenge in FL is the effective aggregation of local model weights from disparate and potentially unbalanced participating clients. Existing methods often treat each client indiscriminately, applying a single proportion to the entire local model. However, it is empirically advantageous for each weight to be assigned a specific proportion. This paper introduces an innovative Element-Wise Weights Aggregation Method for Federated Learning (EWWA-FL) aimed at optimizing learning performance and accelerating convergence speed. Unlike traditional FL approaches, EWWA-FL aggregates local weights to the global model at the level of individual elements, thereby allowing each participating client to make element-wise contributions to the learning process. By taking into account the unique dataset characteristics of each client, EWWA-FL enhances the robustness of the global model to different datasets while also achieving rapid convergence. The method is flexible enough to employ various weighting strategies. Through comprehensive experiments, we demonstrate the advanced capabilities of EWWA-FL, showing significant improvements in both accuracy and convergence speed across a range of backbones and benchmarks.

An Element-Wise Weights Aggregation Method for Federated Learning

TL;DR

EWWA-FL advances federated learning by introducing element-wise aggregation, where each parameter across local models receives a distinct, client-specific weight rather than a single global proportion. Building on FedAvg, it uses Adam-like first- and second-moment estimates to compute per-parameter contributions and applies Softmax to obtain parameter-level aggregation weights, enabling finer alignment to heterogeneous client data. Empirical results show improved accuracy and faster convergence, particularly on large-class datasets and non-IID settings, with high stability and reduced variance compared to peers like FedOpt and FedCAMS. The approach is implemented in an open-source framework, indicating practical applicability and reproducibility for diverse backbone networks and benchmarks.

Abstract

Federated learning (FL) is a powerful Machine Learning (ML) paradigm that enables distributed clients to collaboratively learn a shared global model while keeping the data on the original device, thereby preserving privacy. A central challenge in FL is the effective aggregation of local model weights from disparate and potentially unbalanced participating clients. Existing methods often treat each client indiscriminately, applying a single proportion to the entire local model. However, it is empirically advantageous for each weight to be assigned a specific proportion. This paper introduces an innovative Element-Wise Weights Aggregation Method for Federated Learning (EWWA-FL) aimed at optimizing learning performance and accelerating convergence speed. Unlike traditional FL approaches, EWWA-FL aggregates local weights to the global model at the level of individual elements, thereby allowing each participating client to make element-wise contributions to the learning process. By taking into account the unique dataset characteristics of each client, EWWA-FL enhances the robustness of the global model to different datasets while also achieving rapid convergence. The method is flexible enough to employ various weighting strategies. Through comprehensive experiments, we demonstrate the advanced capabilities of EWWA-FL, showing significant improvements in both accuracy and convergence speed across a range of backbones and benchmarks.
Paper Structure (11 sections, 9 equations, 3 figures, 2 tables, 2 algorithms)

This paper contains 11 sections, 9 equations, 3 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology
  4. FedAvg
  5. Element-Wise Aggregation for FL
  6. Experiments
  7. Settings, Backbones and Datasets
  8. Accuracy on IID data
  9. Accuracy on non-IID data
  10. Convergence Speed
  11. Conclusion

Figures (3)

  • Figure 1: Illustration for our proposed EWWA-FL
  • Figure 2: Diagram of EWWA-FL. The left part is the client that performs the local training. The right side is the server calculating the local contributions and aggregating a new global model based on the local contributions and gradients. All calculations on the server end are done on an element-by-element basis.
  • Figure 3: Visualization of the average training loss across all local clients is shown. The horizontal axis represents the number of global aggregation rounds, while the vertical axis indicates the average loss for the current round. The blue line represents our proposed EWWA-FL, the orange line is from FedCAMS, and the green line represents FedOpt.