Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards Trustworthy Federated Learning

Alina Basharat, Yijun Bian, Ping Xu, Zhi Tian

TL;DR

This work tackles the challenge of trustworthy federated learning by integrating three mechanisms: a fairness-driven objective via $q$-fair FL, a differential privacy scheme to obscure local data, and a two-sided norm-based screening (TNBS) to mitigate Byzantine attacks. The framework defines $H(\theta) = \sum_{i=1}^{M} \frac{p_i}{q+1} F_i^{q+1}(\theta)$ and employs DP-perturbed gradients $\tilde{g}_i = g_i + n$ with $n \sim \mathcal{N}(0,\sigma^2)$, while TNBS keeps the middle $p$-fraction of gradients based on their norms before aggregating to update $\theta$. The authors establish a resilience bound $\|G - \nabla H\| \leq \frac{2\alpha}{1-p} \|\nabla H\| + \max_{i \in M} \|g_i - \nabla H\| + \sigma$ and prove convergence guarantees for both nonconvex and convex cases, showing that the method converges to a stationary point (nonconvex) or achieves a $1/T$-rate (convex) despite DP noise and Byzantine threats. Empirically, the method outperforms baselines under various attacks on MNIST and Spam datasets, while maintaining fairness across clients and demonstrating the expected privacy-accuracy trade-off induced by DP.

Abstract

This paper develops a comprehensive framework to address three critical trustworthy challenges in federated learning (FL): robustness against Byzantine attacks, fairness, and privacy preservation. To improve the system's defense against Byzantine attacks that send malicious information to bias the system's performance, we develop a Two-sided Norm Based Screening (TNBS) mechanism, which allows the central server to crop the gradients that have the l lowest norms and h highest norms. TNBS functions as a screening tool to filter out potential malicious participants whose gradients are far from the honest ones. To promote egalitarian fairness, we adopt the q-fair federated learning (q-FFL). Furthermore, we adopt a differential privacy-based scheme to prevent raw data at local clients from being inferred by curious parties. Convergence guarantees are provided for the proposed framework under different scenarios. Experimental results on real datasets demonstrate that the proposed framework effectively improves robustness and fairness while managing the trade-off between privacy and accuracy. This work appears to be the first study that experimentally and theoretically addresses fairness, privacy, and robustness in trustworthy FL.

Towards Trustworthy Federated Learning

TL;DR

This work tackles the challenge of trustworthy federated learning by integrating three mechanisms: a fairness-driven objective via -fair FL, a differential privacy scheme to obscure local data, and a two-sided norm-based screening (TNBS) to mitigate Byzantine attacks. The framework defines and employs DP-perturbed gradients with , while TNBS keeps the middle -fraction of gradients based on their norms before aggregating to update . The authors establish a resilience bound and prove convergence guarantees for both nonconvex and convex cases, showing that the method converges to a stationary point (nonconvex) or achieves a -rate (convex) despite DP noise and Byzantine threats. Empirically, the method outperforms baselines under various attacks on MNIST and Spam datasets, while maintaining fairness across clients and demonstrating the expected privacy-accuracy trade-off induced by DP.

Abstract

This paper develops a comprehensive framework to address three critical trustworthy challenges in federated learning (FL): robustness against Byzantine attacks, fairness, and privacy preservation. To improve the system's defense against Byzantine attacks that send malicious information to bias the system's performance, we develop a Two-sided Norm Based Screening (TNBS) mechanism, which allows the central server to crop the gradients that have the l lowest norms and h highest norms. TNBS functions as a screening tool to filter out potential malicious participants whose gradients are far from the honest ones. To promote egalitarian fairness, we adopt the q-fair federated learning (q-FFL). Furthermore, we adopt a differential privacy-based scheme to prevent raw data at local clients from being inferred by curious parties. Convergence guarantees are provided for the proposed framework under different scenarios. Experimental results on real datasets demonstrate that the proposed framework effectively improves robustness and fairness while managing the trade-off between privacy and accuracy. This work appears to be the first study that experimentally and theoretically addresses fairness, privacy, and robustness in trustworthy FL.

Paper Structure

This paper contains 13 sections, 3 theorems, 29 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Proposed Method
  3. Theoretical Analysis
  4. Preliminaries
  5. Main theorems
  6. Experiments
  7. Experimental setup
  8. Results
  9. Conclusion
  10. Appendix
  11. Proof of Lemma 1
  12. Proof of Theorem \ref{['thm:theo,1']}
  13. Proof of Theorem \ref{['thm:theo,2']}

Key Result

Lemma 1

Suppose that a percentage of the local gradients, denoted by $\alpha$, are Byzantine and the index set of honest gradients is denoted as $\mathcal{M}$. Additionally, DP is implemented by adding Gaussian noise with variance $\sigma^2$ to each honest gradient. Let $G = \text{TNBS}_p$ be the aggregated where $\nabla H := \frac{1}{M} \sum_{i=1}^{M} \nabla H _i$ is the aggregated gradient and $\nabla H

Figures (4)

  • Figure 1: Trustworthy federated learning.
  • Figure 2: Comparison of model accuracy of proposed method with the other benchmarks for the Spam based dataset.
  • Figure 3: Comparison of model accuracy of proposed method with the other benchmarks for the MNIST dataset.
  • Figure 4: Trade-off between privacy and accuracy.

Theorems & Definitions (4)

  • Definition 1
  • Lemma 1
  • Theorem 1
  • Theorem 2