Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Fast-FedUL: A Training-Free Federated Unlearning with Provable Skew Resilience

Thanh Trung Huynh, Trong Bang Nguyen, Phi Le Nguyen, Thanh Tam Nguyen, Matthias Weidlich, Quoc Viet Hung Nguyen, Karl Aberer

TL;DR

This work tackles federated unlearning by introducing Fast-FedUL, a training-free, server-side method that eliminates a target client’s influence without retraining. It combines a two-stage framework—per-round skew estimation and skew-aware removal—with a memory-efficient update-sampling strategy, leveraging Lipschitz-based bounds to approximate the target's impact. The authors prove theoretical guarantees on the bounded distance to the retrained solution and establish $O(T\times N)$ time complexity, while empirical results on backdoor attacks across multiple datasets show near-retraining utility, negligible backdoor success, and substantial speedups. The approach advances practical FL privacy by enabling efficient, provable unlearning without accessing client data or performing additional training rounds.

Abstract

Federated learning (FL) has recently emerged as a compelling machine learning paradigm, prioritizing the protection of privacy for training data. The increasing demand to address issues such as ``the right to be forgotten'' and combat data poisoning attacks highlights the importance of techniques, known as \textit{unlearning}, which facilitate the removal of specific training data from trained FL models. Despite numerous unlearning methods proposed for centralized learning, they often prove inapplicable to FL due to fundamental differences in the operation of the two learning paradigms. Consequently, unlearning in FL remains in its early stages, presenting several challenges. Many existing unlearning solutions in FL require a costly retraining process, which can be burdensome for clients. Moreover, these methods are primarily validated through experiments, lacking theoretical assurances. In this study, we introduce Fast-FedUL, a tailored unlearning method for FL, which eliminates the need for retraining entirely. Through meticulous analysis of the target client's influence on the global model in each round, we develop an algorithm to systematically remove the impact of the target client from the trained model. In addition to presenting empirical findings, we offer a theoretical analysis delineating the upper bound of our unlearned model and the exact retrained model (the one obtained through retraining using untargeted clients). Experimental results with backdoor attack scenarios indicate that Fast-FedUL effectively removes almost all traces of the target client, while retaining the knowledge of untargeted clients (obtaining a high accuracy of up to 98\% on the main task). Significantly, Fast-FedUL attains the lowest time complexity, providing a speed that is 1000 times faster than retraining. Our source code is publicly available at \url{https://github.com/thanhtrunghuynh93/fastFedUL}.

Fast-FedUL: A Training-Free Federated Unlearning with Provable Skew Resilience

TL;DR

This work tackles federated unlearning by introducing Fast-FedUL, a training-free, server-side method that eliminates a target client’s influence without retraining. It combines a two-stage framework—per-round skew estimation and skew-aware removal—with a memory-efficient update-sampling strategy, leveraging Lipschitz-based bounds to approximate the target's impact. The authors prove theoretical guarantees on the bounded distance to the retrained solution and establish time complexity, while empirical results on backdoor attacks across multiple datasets show near-retraining utility, negligible backdoor success, and substantial speedups. The approach advances practical FL privacy by enabling efficient, provable unlearning without accessing client data or performing additional training rounds.

Abstract

Federated learning (FL) has recently emerged as a compelling machine learning paradigm, prioritizing the protection of privacy for training data. The increasing demand to address issues such as ``the right to be forgotten'' and combat data poisoning attacks highlights the importance of techniques, known as \textit{unlearning}, which facilitate the removal of specific training data from trained FL models. Despite numerous unlearning methods proposed for centralized learning, they often prove inapplicable to FL due to fundamental differences in the operation of the two learning paradigms. Consequently, unlearning in FL remains in its early stages, presenting several challenges. Many existing unlearning solutions in FL require a costly retraining process, which can be burdensome for clients. Moreover, these methods are primarily validated through experiments, lacking theoretical assurances. In this study, we introduce Fast-FedUL, a tailored unlearning method for FL, which eliminates the need for retraining entirely. Through meticulous analysis of the target client's influence on the global model in each round, we develop an algorithm to systematically remove the impact of the target client from the trained model. In addition to presenting empirical findings, we offer a theoretical analysis delineating the upper bound of our unlearned model and the exact retrained model (the one obtained through retraining using untargeted clients). Experimental results with backdoor attack scenarios indicate that Fast-FedUL effectively removes almost all traces of the target client, while retaining the knowledge of untargeted clients (obtaining a high accuracy of up to 98\% on the main task). Significantly, Fast-FedUL attains the lowest time complexity, providing a speed that is 1000 times faster than retraining. Our source code is publicly available at \url{https://github.com/thanhtrunghuynh93/fastFedUL}.
Paper Structure (23 sections, 2 theorems, 46 equations, 9 figures, 4 tables, 2 algorithms)

This paper contains 23 sections, 2 theorems, 46 equations, 9 figures, 4 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation and Design Principles
  4. Fast-FedUL
  5. Overview
  6. Efficient Sampling of Local Updates
  7. Removal of Target Client Updates
  8. Skew Estimation
  9. Theoretical Analysis
  10. Empirical Evaluation
  11. End-to-end Comparison
  12. Ablation Study
  13. Qualitative Study
  14. Conclusion
  15. Appendix
  16. ...and 8 more sections

Key Result

lemma thmcounterlemma

Let $\zeta_1, \zeta_2, \ldots, \zeta_N$ be vectors in $\mathbb{R}^d$ and $w_1, w_2, \ldots, w_N$ be non-negative numbers and $\sum_{i=1}^N w_i=1$, $C$ be a proper sampling. If $v \in \mathbb{R}^N$ is such that then where $\mathbb{E}$ is the expectation taken over C.

Figures (9)

  • Figure 1: Overview of Fast-FedUL. During the learning phase, the server selectively stores the sampled local updates. Upon receiving an unlearning request, the server retrieves the stored updates from memory and performs reverse subtraction using the estimated skew.
  • Figure 2: Unlearning time.
  • Figure 3: Comparison of Fast-FedUL's variants.
  • Figure 4: Robustness against non-IID (edge-case backdoor).
  • Figure 5: Model deviations.
  • ...and 4 more figures

Theorems & Definitions (6)

  • lemma thmcounterlemma
  • proof
  • proof
  • proof
  • lemma thmcounterlemma
  • proof