Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Privacy-preserving gradient-based fair federated learning

Janis Adamek, Moritz Schulze Darup

TL;DR

This paper tackles the problem of achieving collabora­tive fairness in federated learning while preserving privacy against third parties. It introduces GBPPFFL, a gradient-based fair FL framework that relies exclusively on local gradients and leverages homomorphic encryption (CKKS) to enable encrypted aggregation and reward computation. The key contributions include a novel parameterization of gradient-based rewards, a practical encrypted algorithm for gradient aggregation and masking, and a thorough numerical evaluation on MNIST, CIFAR-10, MR, and SST demonstrating comparable accuracy and strong fairness under IID and non-IID data, with manageable computational overhead. The work advances privacy-preserving FL for control-oriented applications and points to future enhancements such as fully encrypted reputation computation and applicability to reinforcement learning settings in industrial contexts.

Abstract

Federated learning (FL) schemes allow multiple participants to collaboratively train neural networks without the need to directly share the underlying data.However, in early schemes, all participants eventually obtain the same model. Moreover, the aggregation is typically carried out by a third party, who obtains combined gradients or weights, which may reveal the model. These downsides underscore the demand for fair and privacy-preserving FL schemes. Here, collaborative fairness asks for individual model quality depending on the individual data contribution. Privacy is demanded with respect to any kind of data outsourced to the third party. Now, there already exist some approaches aiming for either fair or privacy-preserving FL and a few works even address both features. In our paper, we build upon these seminal works and present a novel, fair and privacy-preserving FL scheme. Our approach, which mainly relies on homomorphic encryption, stands out for exclusively using local gradients. This increases the usability in comparison to state-of-the-art approaches and thereby opens the door to applications in control.

Privacy-preserving gradient-based fair federated learning

TL;DR

This paper tackles the problem of achieving collabora­tive fairness in federated learning while preserving privacy against third parties. It introduces GBPPFFL, a gradient-based fair FL framework that relies exclusively on local gradients and leverages homomorphic encryption (CKKS) to enable encrypted aggregation and reward computation. The key contributions include a novel parameterization of gradient-based rewards, a practical encrypted algorithm for gradient aggregation and masking, and a thorough numerical evaluation on MNIST, CIFAR-10, MR, and SST demonstrating comparable accuracy and strong fairness under IID and non-IID data, with manageable computational overhead. The work advances privacy-preserving FL for control-oriented applications and points to future enhancements such as fully encrypted reputation computation and applicability to reinforcement learning settings in industrial contexts.

Abstract

Federated learning (FL) schemes allow multiple participants to collaboratively train neural networks without the need to directly share the underlying data.However, in early schemes, all participants eventually obtain the same model. Moreover, the aggregation is typically carried out by a third party, who obtains combined gradients or weights, which may reveal the model. These downsides underscore the demand for fair and privacy-preserving FL schemes. Here, collaborative fairness asks for individual model quality depending on the individual data contribution. Privacy is demanded with respect to any kind of data outsourced to the third party. Now, there already exist some approaches aiming for either fair or privacy-preserving FL and a few works even address both features. In our paper, we build upon these seminal works and present a novel, fair and privacy-preserving FL scheme. Our approach, which mainly relies on homomorphic encryption, stands out for exclusively using local gradients. This increases the usability in comparison to state-of-the-art approaches and thereby opens the door to applications in control.
Paper Structure (14 sections, 8 equations, 2 figures, 2 tables)

This paper contains 14 sections, 8 equations, 2 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background on fair federated learning and privacy-preserving computations
  3. Fair federated learning via gradient-based rewards
  4. Privacy-preserving computation via homomorphic encryption
  5. Privacy-preserving gradient-based fair federated learning
  6. Some notes on privacy and security
  7. Details on the algorithm
  8. Security issues beyond our model
  9. Numerical Evaluation
  10. Experimental Setup
  11. Case study on the parameterized reward mechanism
  12. Case study on the encrypted implementation
  13. Full experimental results
  14. Conclusion and Outlook

Figures (2)

  • Figure 1: Illustration of (a) the FedSGD scheme and (b) the FFLX scheme, where participants send their local gradients $\Delta \tilde{\boldsymbol{w}}_i^j$ and receive reward gradients $\Delta \widehat{\boldsymbol{w}}_i^j$ based on their reputation $r_i^j$.
  • Figure 2: The two central steps of our GBPPFFL scheme (with details for participant $1$). In (a), the calculation of the encrypted FL gradient and scalar products is depicted. In (b), the encrypted computation of the reward gradients based on the contributions $\phi_i^j$ is sketched.