Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FedUHB: Accelerating Federated Unlearning via Polyak Heavy Ball Method

Yu Jiang, Chee Wei Tan, Kwok-Yan Lam

TL;DR

FedUHB is proposed, a novel exact unlearning approach that leverages the Polyak heavy ball optimization technique, a first-order method, to achieve rapid retraining and introduces a dynamic stopping mechanism to optimize the termination of the unlearning process.

Abstract

Federated learning facilitates collaborative machine learning, enabling multiple participants to collectively develop a shared model while preserving the privacy of individual data. The growing importance of the "right to be forgotten" calls for effective mechanisms to facilitate data removal upon request. In response, federated unlearning (FU) has been developed to efficiently eliminate the influence of specific data from the model. Current FU methods primarily rely on approximate unlearning strategies, which seek to balance data removal efficacy with computational and communication costs, but often fail to completely erase data influence. To address these limitations, we propose FedUHB, a novel exact unlearning approach that leverages the Polyak heavy ball optimization technique, a first-order method, to achieve rapid retraining. In addition, we introduce a dynamic stopping mechanism to optimize the termination of the unlearning process. Our extensive experiments show that FedUHB not only enhances unlearning efficiency but also preserves robust model performance after unlearning. Furthermore, the dynamic stopping mechanism effectively reduces the number of unlearning iterations, conserving both computational and communication resources. FedUHB can be proved as an effective and efficient solution for exact data removal in federated learning settings.

FedUHB: Accelerating Federated Unlearning via Polyak Heavy Ball Method

TL;DR

FedUHB is proposed, a novel exact unlearning approach that leverages the Polyak heavy ball optimization technique, a first-order method, to achieve rapid retraining and introduces a dynamic stopping mechanism to optimize the termination of the unlearning process.

Abstract

Federated learning facilitates collaborative machine learning, enabling multiple participants to collectively develop a shared model while preserving the privacy of individual data. The growing importance of the "right to be forgotten" calls for effective mechanisms to facilitate data removal upon request. In response, federated unlearning (FU) has been developed to efficiently eliminate the influence of specific data from the model. Current FU methods primarily rely on approximate unlearning strategies, which seek to balance data removal efficacy with computational and communication costs, but often fail to completely erase data influence. To address these limitations, we propose FedUHB, a novel exact unlearning approach that leverages the Polyak heavy ball optimization technique, a first-order method, to achieve rapid retraining. In addition, we introduce a dynamic stopping mechanism to optimize the termination of the unlearning process. Our extensive experiments show that FedUHB not only enhances unlearning efficiency but also preserves robust model performance after unlearning. Furthermore, the dynamic stopping mechanism effectively reduces the number of unlearning iterations, conserving both computational and communication resources. FedUHB can be proved as an effective and efficient solution for exact data removal in federated learning settings.

Paper Structure

This paper contains 18 sections, 1 theorem, 13 equations, 5 figures, 1 table, 1 algorithm.

Table of Contents

  1. Introduction
  2. FedUHB Scheme
  3. Federated Learning
  4. Federated Unlearning
  5. Theoretic Analysis
  6. Experiment
  7. Evaluation Setup
  8. Compared methods
  9. Implementation details
  10. Evaluation metrics
  11. Datasets and models
  12. Evaluation Results and Analysis
  13. Running time
  14. Convergence rate of unlearning
  15. Accuracy on the test data
  16. ...and 3 more sections

Key Result

Theorem 1

The upper bound of the model difference between FedUHB and retrain after $t$ rounds can be expressed as: where $\bar{w}_{t}$ and $\hat{w}_{t}$ are the global models unlearned by FedUHB and the retrain method after $t$ rounds, respectively.

Figures (5)

  • Figure 1: An illustration of FedUHB Scheme. FedUHB is an exact unlearning method that accelerates convergence using Polyak heavy ball optimization and features a dynamic stopping mechanism to optimize the termination of the unlearning process.
  • Figure 2: Loss curve of the unlearning on MNIST and CIFAR-10
  • Figure 3: Test accuracy of the unlearning on MNIST and CIFAR-10
  • Figure 4: MISR (Bar) and ASR (Line) on MNIST and CIFAR-10
  • Figure 5: Impact of parameters on MNIST

Theorems & Definitions (2)

  • Theorem 1
  • proof