FedAA: A Reinforcement Learning Perspective on Adaptive Aggregation for Fair and Robust Federated Learning

TL;DR

The paper tackles robustness and fairness in Federated Learning under non-IID data and adversarial clients by introducing FedAA, which uses Deep Deterministic Policy Gradient to continuously adapt aggregation weights. The server acts as a DRL agent that selects a top-$\text{M%}$ subset of clients based on model-distance and outputs aggregation weights, with rewards derived from accuracy on a server-held fair validation set. Empirical results show FedAA achieves state-of-the-art robustness under multiple attack models while maintaining comparable fairness to existing baselines, and it demonstrates a tunable trade-off between robustness and fairness via the participating fraction. This work integrates DRL into the server-side aggregation process to yield resilient and fair federated systems, with practical implications for deployment in heterogeneous and potentially adversarial environments.

Abstract

Federated Learning (FL) has emerged as a promising approach for privacy-preserving model training across decentralized devices. However, it faces challenges such as statistical heterogeneity and susceptibility to adversarial attacks, which can impact model robustness and fairness. Personalized FL attempts to provide some relief by customizing models for individual clients. However, it falls short in addressing server-side aggregation vulnerabilities. We introduce a novel method called \textbf{FedAA}, which optimizes client contributions via \textbf{A}daptive \textbf{A}ggregation to enhance model robustness against malicious clients and ensure fairness across participants in non-identically distributed settings. To achieve this goal, we propose an approach involving a Deep Deterministic Policy Gradient-based algorithm for continuous control of aggregation weights, an innovative client selection method based on model parameter distances, and a reward mechanism guided by validation set performance. Empirically, extensive experiments demonstrate that, in terms of robustness, \textbf{FedAA} outperforms the state-of-the-art methods, while maintaining comparable levels of fairness, offering a promising solution to build resilient and fair federated systems. Our code is available at https://github.com/Gp1g/FedAA.

Figures (5)

• Figure 1: The figures in the first line represent robustness performance (i.e. mean test accuracy across benign clients) of three different datasets subjected to three different attacks. The figures in the second line depict the performance of three different datasets with no malicious clients.
• Figure 2: The tradeoff between test accuracy and fairness within different methods. The closer the approach is to the lower right corner, the better.
• Figure 3: The performance and tradeoff between robustness and fairness of different $\rm M$ (The numbers on the x-axis in the figures represent the corresponding $\rm M\%$, e.g. 80 means $\rm M=80\%$).
• Figure 4: The convergence curve of reward $r$ of each class at the server.
• Figure 5: The tradeoff between test accuracy and fairness under the same value attack. The closer the $\rm M$ is to the lower right corner, the better. (The numbers in the figure correspond to the percentages participating in aggregation, e.g. 80 means $\rm M=80\%$)