Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedDRL: A Trustworthy Federated Learning Model Fusion Method Based on Staged Reinforcement Learning

Leiming Chen, Weishan Zhang, Cihao Dong, Sibo Qiao, Ziling Huang, Yuming Nie, Zhaoxiang Hou, Chee Wei Tan

TL;DR

This work tackles the vulnerability of federated learning to malicious and low-quality client models under non-IID data. It introduces FedDRL, a staged reinforcement learning framework that first filters trustworthy clients via distributed A2C and then adaptively assigns fusion weights using TD3, redefining aggregation from fixed sample-size weights to RL-driven weights. The authors validate FedDRL across five model-fusion scenarios and multiple datasets, demonstrating improved reliability and maintained accuracy compared with FedAvg and FedProx, even in adversarial conditions. They also show that a multi-agent setup can accelerate agent training, offering a practical pathway toward robust and scalable trustworthy federated learning deployment.

Abstract

Traditional federated learning uses the number of samples to calculate the weights of each client model and uses this fixed weight value to fusion the global model. However, in practical scenarios, each client's device and data heterogeneity leads to differences in the quality of each client's model. Thus the contribution to the global model is not wholly determined by the sample size. In addition, if clients intentionally upload low-quality or malicious models, using these models for aggregation will lead to a severe decrease in global model accuracy. Traditional federated learning algorithms do not address these issues. To solve this probelm, we propose FedDRL, a model fusion approach using reinforcement learning based on a two staged approach. In the first stage, Our method could filter out malicious models and selects trusted client models to participate in the model fusion. In the second stage, the FedDRL algorithm adaptively adjusts the weights of the trusted client models and aggregates the optimal global model. We also define five model fusion scenarios and compare our method with two baseline algorithms in those scenarios. The experimental results show that our algorithm has higher reliability than other algorithms while maintaining accuracy.

FedDRL: A Trustworthy Federated Learning Model Fusion Method Based on Staged Reinforcement Learning

TL;DR

This work tackles the vulnerability of federated learning to malicious and low-quality client models under non-IID data. It introduces FedDRL, a staged reinforcement learning framework that first filters trustworthy clients via distributed A2C and then adaptively assigns fusion weights using TD3, redefining aggregation from fixed sample-size weights to RL-driven weights. The authors validate FedDRL across five model-fusion scenarios and multiple datasets, demonstrating improved reliability and maintained accuracy compared with FedAvg and FedProx, even in adversarial conditions. They also show that a multi-agent setup can accelerate agent training, offering a practical pathway toward robust and scalable trustworthy federated learning deployment.

Abstract

Traditional federated learning uses the number of samples to calculate the weights of each client model and uses this fixed weight value to fusion the global model. However, in practical scenarios, each client's device and data heterogeneity leads to differences in the quality of each client's model. Thus the contribution to the global model is not wholly determined by the sample size. In addition, if clients intentionally upload low-quality or malicious models, using these models for aggregation will lead to a severe decrease in global model accuracy. Traditional federated learning algorithms do not address these issues. To solve this probelm, we propose FedDRL, a model fusion approach using reinforcement learning based on a two staged approach. In the first stage, Our method could filter out malicious models and selects trusted client models to participate in the model fusion. In the second stage, the FedDRL algorithm adaptively adjusts the weights of the trusted client models and aggregates the optimal global model. We also define five model fusion scenarios and compare our method with two baseline algorithms in those scenarios. The experimental results show that our algorithm has higher reliability than other algorithms while maintaining accuracy.
Paper Structure (23 sections, 22 equations, 9 figures, 7 tables, 3 algorithms)

This paper contains 23 sections, 22 equations, 9 figures, 7 tables, 3 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Federated Learning
  4. Challenges of Non-IID Data Distribution
  5. Federated Reinforcement Learning
  6. Method
  7. Problem Definition
  8. A Trustworthy Federated Learning Approach Based on Staged Reinforcement Learning
  9. Adaptive client selection method
  10. Adaptive model weight calculation method
  11. SYSTEM DESIGN
  12. EXPERIMENT
  13. Experiment setup
  14. Experiment datasets
  15. Comparison of Methods
  16. ...and 8 more sections

Figures (9)

  • Figure 1: The Process of FedDRL framework
  • Figure 2: The system architecture of FedDRL
  • Figure 3: The non-iid distribution of 10 clients(alpha=1).
  • Figure 4: The accuracy of the global model for different number of client in attack type 1
  • Figure 5: The accuracy of the global model for different number of client in attack type 2
  • ...and 4 more figures