NoT: Federated Unlearning via Weight Negation

TL;DR

NoT introduces federated unlearning via weight negation, a storage-free perturbation that disrupts inter-layer co-adaptation and enables rapid recovery through fine-tuning on retained data. The authors provide a theoretical framework linking strong perturbations, Jacobian control, and layer-wise optimality to a practical two-step FU process: negate selected layer weights to create a perturbed model, then fine-tune on retained data to forget target data without access to it. Empirically, NoT outperforms seven FU baselines across CIFAR-10/100 and Caltech-101 with CNN, ResNet-18, and ViT architectures, including backdoor mitigation and centralized settings, while incurring minimal communication and computation costs. The work further substantiates its claims with extensive ablations, CKA analyses, and theoretical results on unlearning time, activation distance, and Jacobian behavior, highlighting NoT’s robustness, efficiency, and practical applicability for privacy-preserving learning. Overall, NoT offers a principled, scalable solution for FU that does not require the remembered data and significantly reduces the overhead associated with data forgetting in federated environments.

Abstract

Federated unlearning (FU) aims to remove a participant's data contributions from a trained federated learning (FL) model, ensuring privacy and regulatory compliance. Traditional FU methods often depend on auxiliary storage on either the client or server side or require direct access to the data targeted for removal-a dependency that may not be feasible if the data is no longer available. To overcome these limitations, we propose NoT, a novel and efficient FU algorithm based on weight negation (multiplying by -1), which circumvents the need for additional storage and access to the target data. We argue that effective and efficient unlearning can be achieved by perturbing model parameters away from the set of optimal parameters, yet being well-positioned for quick re-optimization. This technique, though seemingly contradictory, is theoretically grounded: we prove that the weight negation perturbation effectively disrupts inter-layer co-adaptation, inducing unlearning while preserving an approximate optimality property, thereby enabling rapid recovery. Experimental results across three datasets and three model architectures demonstrate that NoT significantly outperforms existing baselines in unlearning efficacy as well as in communication and computational efficiency.

Figures (8)

• Figure 1: Performance comparison of NoT with baselines using ViT-B/16 on Caltech-101 in a 10-client setup, where one client requests unlearning. The ideal federated unlearning algorithm should closely approximate the performance of the “gold standard” (Retrain) across key accuracy metrics: retain, forget, test, and MIA, while minimizing communication and computation overhead. As illustrated, NoT's performance closely matches that of Retrain across all metrics with minimal added costs, underscoring NoT’s efficacy and efficiency in federated unlearning. Experimental details and further comparisons can be found in Section \ref{['sec_experimens']}.
• Figure 2: NoT overview. Upon receiving unlearning requests from target clients, the server initiates the unlearning process by applying layer-wise parameter negation to the global model. This negation disrupts inter-layer co-adaptation, effectively inducing unlearning. Subsequent fine-tuning rounds restore essential knowledge. If a client wishes to forget all its data (i.e., client-wise forgetting), it does not participate in fine-tuning. Conversely, if a client wants partial data forgetting (i.e., class-wise or instance-wise forgetting), it fine-tunes the global model using its retained data.
• Figure 3: CKA of layer activations for various models compared to the original model ($\theta=\theta^*$) before fine-tuning (FT@$\tau$:0). The first and last communication rounds are denoted by $\tau$:0 and -1. $(-\theta_0)\oplus \left( \mathrm{R}(\theta_\ell)\right)_{\ell\neq 0}$ denotes a model with negated first-layer weights ($\ell$:0) and randomized rest $R(\cdot)$.
• Figure 4: Effect of negating different ViT layers. Negating the convolution projection layer resulted in the best unlearning performance.
• Figure 5: Effect of different perturbations on ViT convolution projection layer. Applying weight negation is the best perturbation for inducing unlearning.

Theorems & Definitions (33)

• Definition 1: Loss gap
• Theorem 1
• Definition 2: Layer-Wise Optimality
• Theorem 2
• Theorem 3
• Theorem 4
• Definition 3: Grafting
• Definition 4: Co-Adapted Layers
• Theorem 5
• proof