Streamlined Federated Unlearning: Unite as One to Be Highly Efficient

TL;DR

SFU tackles privacy-driven unlearning in federated learning by unifying influence removal and performance preservation into a single distillation-based step. It employs a three-teacher system—$M^P$ (performance preservation), $M^F$ (forgetting), and $M_i^L$ (label-based preservation)—to guide a local unlearned model $M_i^U$ via KL-divergence objectives while keeping data local. Theoretical analysis shows target-data predictions converge toward a uniform distribution while retained-data predictions stay aligned with the original model and true labels, and experiments on image and text benchmarks show SFU achieves comparable or better unlearning efficacy than retraining, with far better time and communication efficiency. SFU preserves privacy by avoiding raw data sharing and integrates non-disruptively into existing FL workflows, demonstrating strong generalization across model types and unlearning scenarios, including backdoor verification.

Abstract

Recently, the enactment of ``right to be forgotten" laws and regulations has imposed new privacy requirements on federated learning (FL). Researchers aim to remove the influence of certain data from the trained model without training from scratch through federated unlearning (FU). While current FU research has shown progress in enhancing unlearning efficiency, it often results in degraded model performance upon achieving the goal of data unlearning, necessitating additional steps to recover the performance of the unlearned model. Moreover, these approaches also suffer from many shortcomings such as high consumption of computational and storage resources. To this end, we propose a streamlined federated unlearning approach (SFU) aimed at effectively removing the influence of the target data while preserving the model performance on the retained data without degradation. We design a practical multi-teacher system that achieves both target data influence removal and model performance preservation by guiding the unlearned model through several distinct teacher models. SFU is both computationally and storage-efficient, highly flexible, and generalizable. We conduct extensive experiments on both image and text benchmark datasets. The results demonstrate that SFU significantly improves time and communication efficiency compared to the benchmark retraining method and significantly outperforms existing SOTA methods. Additionally, we verify the effectiveness of SFU using the backdoor attack.

Figures (5)

• Figure 1: The general process of federated unlearning.
• Figure 2: The framework of SFU. When an unlearning request reaches the server, each client (client $i$, for example) that has the need for the forgotten data carries out the unlearning process in the figure through the multi-teacher system. After completing the unlearning, the corresponding client deletes the forgotten data locally and resumes the normal FL process.
• Figure 3: Test accuracy of models using different unlearning methods on forgotten and retained data. Three different classes of data on each of the three different datasets are randomly selected to be unlearned using different methods. These data are randomly distributed on different clients according to non-iid. The lines of different colors represent the unlearning effects of different methods. The numbers of different colors on the horizontal axis represent the number of communication rounds required by each method to complete unlearning.
• Figure 4: The unlearning performance of SFU under different combinations of teacher models. The lines of different colors represent the unlearning effects of different combinations of teacher models
• Figure 5: Comparison of the unlearning performance of SFU with different values of hyperparameters $\alpha$. We set three different orders of magnitude of $\alpha$ to compare its impact on unlearning: $\alpha$ calculated according to the proposed method and $\alpha$ one order of magnitude above and below it. We select one class of data from each of the three datasets for unlearning.