Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Unlearning during Learning: An Efficient Federated Machine Unlearning Method

Hanlin Gu, Gongxi Zhu, Jie Zhang, Xinyuan Zhao, Yuxing Han, Lixin Fan, Qiang Yang

TL;DR

FedAU is introduced, an innovative and efficient FMU framework that incorporates a lightweight auxiliary unlearning module into the learning process and employs a straightforward linear operation to facilitate unlearning.

Abstract

In recent years, Federated Learning (FL) has garnered significant attention as a distributed machine learning paradigm. To facilitate the implementation of the right to be forgotten, the concept of federated machine unlearning (FMU) has also emerged. However, current FMU approaches often involve additional time-consuming steps and may not offer comprehensive unlearning capabilities, which renders them less practical in real FL scenarios. In this paper, we introduce FedAU, an innovative and efficient FMU framework aimed at overcoming these limitations. Specifically, FedAU incorporates a lightweight auxiliary unlearning module into the learning process and employs a straightforward linear operation to facilitate unlearning. This approach eliminates the requirement for extra time-consuming steps, rendering it well-suited for FL. Furthermore, FedAU exhibits remarkable versatility. It not only enables multiple clients to carry out unlearning tasks concurrently but also supports unlearning at various levels of granularity, including individual data samples, specific classes, and even at the client level. We conducted extensive experiments on MNIST, CIFAR10, and CIFAR100 datasets to evaluate the performance of FedAU. The results demonstrate that FedAU effectively achieves the desired unlearning effect while maintaining model accuracy. Our code is availiable at https://github.com/Liar-Mask/FedAU.

Unlearning during Learning: An Efficient Federated Machine Unlearning Method

TL;DR

FedAU is introduced, an innovative and efficient FMU framework that incorporates a lightweight auxiliary unlearning module into the learning process and employs a straightforward linear operation to facilitate unlearning.

Abstract

In recent years, Federated Learning (FL) has garnered significant attention as a distributed machine learning paradigm. To facilitate the implementation of the right to be forgotten, the concept of federated machine unlearning (FMU) has also emerged. However, current FMU approaches often involve additional time-consuming steps and may not offer comprehensive unlearning capabilities, which renders them less practical in real FL scenarios. In this paper, we introduce FedAU, an innovative and efficient FMU framework aimed at overcoming these limitations. Specifically, FedAU incorporates a lightweight auxiliary unlearning module into the learning process and employs a straightforward linear operation to facilitate unlearning. This approach eliminates the requirement for extra time-consuming steps, rendering it well-suited for FL. Furthermore, FedAU exhibits remarkable versatility. It not only enables multiple clients to carry out unlearning tasks concurrently but also supports unlearning at various levels of granularity, including individual data samples, specific classes, and even at the client level. We conducted extensive experiments on MNIST, CIFAR10, and CIFAR100 datasets to evaluate the performance of FedAU. The results demonstrate that FedAU effectively achieves the desired unlearning effect while maintaining model accuracy. Our code is availiable at https://github.com/Liar-Mask/FedAU.
Paper Structure (50 sections, 4 theorems, 22 equations, 7 figures, 9 tables, 4 algorithms)

This paper contains 50 sections, 4 theorems, 22 equations, 7 figures, 9 tables, 4 algorithms.

Table of Contents

  1. Introduction
  2. Relate Work
  3. Federated Learning
  4. Machine Unlearning
  5. The Proposed Method
  6. Setting
  7. FedAU
  8. Learning Module
  9. Auxiliary Unlearning Module
  10. Linear Operation on $W^l$ and $W^a$
  11. Generality of FedAU
  12. Unlearning Sample in FL
  13. Unlearning Class in FL
  14. Unlearning a Client in FL
  15. Unlearning for Multiple Clients
  16. ...and 35 more sections

Key Result

Proposition 1

Consider two fully connected layers projecting the input $x \in \mathbb{R}^{m_2}$ to the logit $l \in \mathbb{R}^{m_1}$ as: $l_1= w_1x + b_1, l_2 = w_2x +b_2$, then the linear operation of weights $w_1, b_1$ and $w_2, b_2$ has the same influence on logits $l_1$ and $l_2$.

Figures (7)

  • Figure 1: Left: the scenario of federated machine unlearning; Right: the overview of the proposed FedAU consisted of three modules/operations (blue: learning module, green: auxiliary unlearning module and red: linear operation).
  • Figure 2: Illustration of the proposed FedAU when unlearning sample and class. After the module $W^l$ undergoes linear operation with the auxiliary unlearning module $W^a$, the unlearned part of the original feature will be classified into other random classes.
  • Figure 3: The accuracy of FedAU and retraining methods on CIFAR10 with different number of unlearning clients.
  • Figure 4: The impact of Non-IID on CIFAR10 for the proposed FedAU and Retraining methods.
  • Figure 5: The impact of coefficient $\alpha$ and $\beta$ for the proposed FedAU method on CIFAR10.
  • ...and 2 more figures

Theorems & Definitions (11)

  • Remark 1
  • Proposition 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Theorem 1
  • Remark 5
  • Proposition 2
  • proof
  • Theorem 2
  • ...and 1 more