FedQUIT: On-Device Federated Unlearning via a Quasi-Competent Virtual Teacher

TL;DR

FedQUIT addresses the challenge of unlearning in cross-device federated learning by enabling on-device data removal through a quasi-competent virtual teacher. It uses a knowledge-distillation framework where a modified global model guides the local unlearned model on forget data, balancing forgetting with retention of generalization without relying on historical updates or proxy data. Empirical results on CIFAR-10/100 demonstrate that FedQUIT achieves forgetting close to a retrained gold standard while significantly reducing cumulative communication costs, and it outperforms state-of-the-art FU baselines like PGA. The approach is analyzed for sensitivity to the modification parameter and robustness to fine-tuning, and is presented as a practical, privacy-preserving solution for federated unlearning across non-IID data and limited-device settings.

Abstract

Federated Learning (FL) systems enable the collaborative training of machine learning models without requiring centralized collection of individual data. FL participants should have the ability to exercise their right to be forgotten, ensuring their past contributions can be removed from the learned model upon request. In this paper, we propose FedQUIT, a novel algorithm that uses knowledge distillation to scrub the contribution of the data to forget from an FL global model while preserving its generalization ability. FedQUIT directly works on client devices that request to leave the federation, and leverages a teacher-student framework. The FL global model acts as the teacher, and the local model works as the student. To induce forgetting, FedQUIT tailors the teacher's output on local data (the data to forget) penalizing the prediction score of the true class. Unlike previous work, our method does not require hardly viable assumptions for cross-device settings, such as storing historical updates of participants or requiring access to proxy datasets. Experimental results on various datasets and model architectures demonstrate that (i) FedQUIT outperforms state-of-the-art competitors in forgetting data, (ii) has the exact computational requirements as a regular FedAvg round, and (iii) reduces the cumulative communication costs by up to 117.6$\times$ compared to retraining from scratch to restore the initial generalization performance after unlearning.

Figures (10)

• Figure 1: FedQUIT overview. (1) Regular training via FedAvg is performed. (2) A client $u$ submits an unlearning request to the server. (3) The server initiates a special round, in which only client $u$ participates, and runs FedQUIT locally starting from the current global model. Client $u$ sends back the unlearned model $w^{\bar{u}}$. (4) Regular training resumes from the unlearned model $w^{\bar{u}}$.
• Figure 2: FedQUIT-Logits.
• Figure 3: Test accuracy degradation after unlearning a client's data from the FL global model, before recovery begins. Lower values indicate better preservation of generalization ability.
• Figure 4: Visualization of FedQUIT and PGA performance across settings. A smaller polygon indicates better unlearning effectiveness.
• Figure 5: Test Accuracy (Left) and Forget Accuracy (Right) for a representative client on CIFAR-100, ResNet-18, Non-IID, $E=1$.