Privacy Preserving and Robust Aggregation for Cross-Silo Federated Learning in Non-IID Settings

TL;DR

This work tackles privacy and robustness in cross-silo federated learning under non-IID data by introducing class-aware gradient masking that relies solely on gradient updates, eliminating metadata leakage. The method assigns a dominant class to each client model using class-specific validation, derives class-relevant gradient masks, and aggregates masked models with a weight proportional to retained parameters. Empirical results show substantial accuracy gains over FedAvg and other baselines across CIFAR-10/100 and FashionMNIST under Dirichlet non-IID settings, while also reducing vulnerability to backdoor and convergence-prevention attacks. The approach achieves privacy preservation with gradient-only aggregation, at the cost of higher server-side computation and slower convergence, but with improved final performance and attack resilience in realistic cross-silo deployments.

Abstract

Federated Averaging remains the most widely used aggregation strategy in federated learning due to its simplicity and scalability. However, its performance degrades significantly in non-IID data settings, where client distributions are highly imbalanced or skewed. Additionally, it relies on clients transmitting metadata, specifically the number of training samples, which introduces privacy risks and may conflict with regulatory frameworks like the European GDPR. In this paper, we propose a novel aggregation strategy that addresses these challenges by introducing class-aware gradient masking. Unlike traditional approaches, our method relies solely on gradient updates, eliminating the need for any additional client metadata, thereby enhancing privacy protection. Furthermore, our approach validates and dynamically weights client contributions based on class-specific importance, ensuring robustness against non-IID distributions, convergence prevention, and backdoor attacks. Extensive experiments on benchmark datasets demonstrate that our method not only outperforms FedAvg and other widely accepted aggregation strategies in non-IID settings but also preserves model integrity in adversarial scenarios. Our results establish the effectiveness of gradient masking as a practical and secure solution for federated learning.

Figures (4)

• Figure 1: An overview of the workflow for our proposed approach within a federated learning network. Local clients train the current global model using their own data, after which the updates are sent to the trusted aggregator. The participating clients can be either benign (BC) or malicious (MC). The process begins with the aggregator evaluating client models using its test set to determine their performance across different classes. Based on this evaluation, the aggregator assigns a dominant class to each model and computes gradients relative to the assigned class. A mask is then generated from these gradients and updated by summing it with the mask from the previous round. The updated mask is applied to the model, filtering out less relevant parameters. Finally, the aggregator performs weighted averaging over the masked models, aggregating contributions from both benign and potentially malicious clients, to generate the new global model.
• Figure 2: Comparison of class distributions across clients in the federated Fashion-MNIST dataset for different values of $\alpha$.
• Figure 3: Accuracy progression over training rounds for different aggregation methods on CIFAR-10. The experiment follows a Dirichlet distribution with $\alpha = 0.3$, using 10 clients
• Figure :