Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ConDa: Fast Federated Unlearning with Contribution Dampening

Vikram S Chundawat, Pushkar Niroula, Prasanna Dhungana, Stefan Schoepf, Murari Mandal, Alexandra Brintrup

TL;DR

Contribution Dampening (ConDa) is introduced, a framework that performs efficient unlearning by tracking down the parameters which affect the global model for each client and performs synaptic dampening on the parameters of the global model that have privacy infringing contributions from the forgetting client.

Abstract

Federated learning (FL) has enabled collaborative model training across decentralized data sources or clients. While adding new participants to a shared model does not pose great technical hurdles, the removal of a participant and their related information contained in the shared model remains a challenge. To address this problem, federated unlearning has emerged as a critical research direction, seeking to remove information from globally trained models without harming the model performance on the remaining data. Most modern federated unlearning methods use costly approaches such as the use of remaining clients data to retrain the global model or methods that would require heavy computation on client or server side. We introduce Contribution Dampening (ConDa), a framework that performs efficient unlearning by tracking down the parameters which affect the global model for each client and performs synaptic dampening on the parameters of the global model that have privacy infringing contributions from the forgetting client. Our technique does not require clients data or any kind of retraining and it does not put any computational overhead on either the client or server side. We perform experiments on multiple datasets and demonstrate that ConDa is effective to forget a client's data. In experiments conducted on the MNIST, CIFAR10, and CIFAR100 datasets, ConDa proves to be the fastest federated unlearning method, outperforming the nearest state of the art approach by at least 100x. Our emphasis is on the non-IID Federated Learning setting, which presents the greatest challenge for unlearning. Additionally, we validate ConDa's robustness through backdoor and membership inference attacks. We envision this work as a crucial component for FL in adhering to legal and ethical requirements.

ConDa: Fast Federated Unlearning with Contribution Dampening

TL;DR

Contribution Dampening (ConDa) is introduced, a framework that performs efficient unlearning by tracking down the parameters which affect the global model for each client and performs synaptic dampening on the parameters of the global model that have privacy infringing contributions from the forgetting client.

Abstract

Federated learning (FL) has enabled collaborative model training across decentralized data sources or clients. While adding new participants to a shared model does not pose great technical hurdles, the removal of a participant and their related information contained in the shared model remains a challenge. To address this problem, federated unlearning has emerged as a critical research direction, seeking to remove information from globally trained models without harming the model performance on the remaining data. Most modern federated unlearning methods use costly approaches such as the use of remaining clients data to retrain the global model or methods that would require heavy computation on client or server side. We introduce Contribution Dampening (ConDa), a framework that performs efficient unlearning by tracking down the parameters which affect the global model for each client and performs synaptic dampening on the parameters of the global model that have privacy infringing contributions from the forgetting client. Our technique does not require clients data or any kind of retraining and it does not put any computational overhead on either the client or server side. We perform experiments on multiple datasets and demonstrate that ConDa is effective to forget a client's data. In experiments conducted on the MNIST, CIFAR10, and CIFAR100 datasets, ConDa proves to be the fastest federated unlearning method, outperforming the nearest state of the art approach by at least 100x. Our emphasis is on the non-IID Federated Learning setting, which presents the greatest challenge for unlearning. Additionally, we validate ConDa's robustness through backdoor and membership inference attacks. We envision this work as a crucial component for FL in adhering to legal and ethical requirements.
Paper Structure (10 sections, 10 equations, 7 figures, 5 tables, 1 algorithm)

This paper contains 10 sections, 10 equations, 7 figures, 5 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Proposed Method
  4. Experiments and Results
  5. Experimental Setup
  6. Results
  7. Ablation Analysis
  8. Conclusion
  9. Appendix
  10. Related Work

Figures (7)

  • Figure 1: The process of Client unlearning in a federated learning (FL) setting is depicted. We also show the efficient nature of the proposed ConDa for federated unlearning.
  • Figure 2: The proposed Contributed Dampening (ConDa) method for federated unlearning.
  • Figure 3: ConDa Federated Unlearning
  • Figure 4: We show the results of ConDa on several Unlearning metrics: Accuracy (R-Set), Accuracy (U-Set), Backdoor Attack, and MI attack at different cut-off ratio $\alpha$. We visualize the unlearning plateau where R-Set accuracy, U-Set accuracy, Backdoor attack and MIA are near ideal values. Setting the $\alpha$ above or below the plateau leads to drop in desired unlearning performance. Results are shown in the order, CIFAR-10, MNIST, CIFAR-100 (left-to-right).
  • Figure 5: We present the results of ConDa for unlearning various clients (client 1 - client 9 in CIFAR-10) from the global model. These results are compared with the retrained model, which serves as the ground truth for unlearning. The performance of ConDa at different cut-off ratios $\alpha$ is displayed, with the optimal trade-off highlighted in the graph.
  • ...and 2 more figures

Theorems & Definitions (1)

  • Definition 1