FedMUA: Exploring the Vulnerabilities of Federated Learning to Malicious Unlearning Attacks

TL;DR

FedMUA identifies a previously overlooked security risk in federated unlearning by showing that an adversarial client can steer the unlearning process to bias a target's prediction while preserving others. The approach combines Influential Sample Identification with Malicious Unlearning Generation to craft constrained feature perturbations that push the attack toward the target data, requiring only black-box access. A lightweight gradient-based defense is proposed to detect and attenuate such malicious unlearning requests, mitigating their impact. Extensive experiments across five datasets and multiple FU methods demonstrate high attack effectiveness and show how defense and data conditions influence the risk, highlighting the need for robust unlearning protocols in FL systems.

Abstract

Recently, the practical needs of ``the right to be forgotten'' in federated learning gave birth to a paradigm known as federated unlearning, which enables the server to forget personal data upon the client's removal request. Existing studies on federated unlearning have primarily focused on efficiently eliminating the influence of requested data from the client's model without retraining from scratch, however, they have rarely doubted the reliability of the global model posed by the discrepancy between its prediction performance before and after unlearning. To bridge this gap, we take the first step by introducing a novel malicious unlearning attack dubbed FedMUA, aiming to unveil potential vulnerabilities emerging from federated learning during the unlearning process. The crux of FedMUA is to mislead the global model into unlearning more information associated with the influential samples for the target sample than anticipated, thus inducing adverse effects on target samples from other clients. To achieve this, we design a novel two-step method, known as Influential Sample Identification and Malicious Unlearning Generation, to identify and subsequently generate malicious feature unlearning requests within the influential samples. By doing so, we can significantly alter the predictions pertaining to the target sample by initiating the malicious feature unlearning requests, leading to the deliberate manipulation for the user adversely. Additionally, we design a new defense mechanism that is highly resilient against malicious unlearning attacks. Extensive experiments on three realistic datasets reveal that FedMUA effectively induces misclassification on target samples and can achieve an 80% attack success rate by triggering only 0.3% malicious unlearning requests.

Figures (10)

• Figure 1: Overview of FedMUA. Taking credit rating as an example, several clients collaboratively train a global model. A particular client acts as an attacker, aiming to submit malicious unlearning requests to the server. The server will receive malicious unlearning gradient updates from this client. After updating these gradients on the global model and each local model, the target user initially identified with a high credit rating from other clients is successfully misclassified as having a low credit rating.
• Figure 2: An illustration of FedMUA in the IID setting. The ovals in different colors denote different classes. The solid line represents the decision boundary and the dotted line indicates the decision boundary after unlearning. Moving the unlearned data to the decision boundary for unlearning can substantially alter the decision boundary around the target sample.
• Figure 3: Key observation in MNIST. The value of gradients refers to the sum of the $\ell_2$-norms of each gradient for each client in each training round. Specifically, client 0 and client 1 send malicious unlearning requests to the server. By visualizing the gradient updates for each client in each training round, we observe that the gradient updates from client 0 and client 1 are generally larger than those from normal clients in the initial training rounds.
• Figure 4: ASR of FedMUA and baseline on FedEraser.
• Figure 6: ASR of FedMUA for attacking different numbers of multi-targets.