FedSGT: Exact Federated Unlearning via Sequential Group-based Training

TL;DR

This work addresses the challenge of exact unlearning in federated learning by introducing FedSGT, which partitions client data into uniform groups and trains multiple lightweight PEFT modules sequentially across several group permutations. Exact unlearning is achieved by deactivating the modules linked to the deleted data, avoiding retraining, while multiple training sequences preserve model utility under accumulating deletions. The authors provide theoretical analyses of deletion rate, performance guarantees, and overhead, and validate FedSGT empirically across vision and language tasks, showing longer service maintenance with comparable accuracy and efficiency to strong baselines. Ablation studies confirm robustness to data heterogeneity and parameter choices, demonstrating FedSGT’s practical viability for compliant, privacy-preserving FL with repeated unlearning requests.

Abstract

Federated Learning (FL) enables collaborative, privacy-preserving model training, but supporting the "Right to be Forgotten" is especially challenging because data influences the model through distributed and interleaved client updates. Existing exact unlearning methods typically require frequent retraining from scratch, resulting in high communication cost and long service downtime. To address this, we propose Federated Sequential Group-based Training (FedSGT), an exact unlearning framework for FL. FedSGT partitions the data into uniform groups, and each client may participate in multiple groups. To control communication overhead, each client can limit the number of groups it contributes to. FedSGT then trains multiple sequences of Parameter-Efficient Fine-Tuning (PEFT) modules, each corresponding to a different group permutation. Since the PEFT modules are lightweight and maintained server-side, FedSGT isolates the influence of different data groups into independent modules without incurring significant storage overhead and communication cost. Exact unlearning is thus achieved instantly by deactivating the modules corresponding to the group containing the unlearned data. Furthermore, using multiple training sequences helps maintain high model utility as deletion requests accumulate. We provide a rigorous theoretical analysis of both the deletion rate -- expected number of deletions before retraining is needed -- and the expected model performance. Experiments on various tasks demonstrate that FedSGT achieves a significantly longer service maintenance under multiple unlearning requests while maintaining comparable learning performance and training efficiency to other exact unlearning baselines. Extensive ablation studies validate the robustness of our method across a wide range of parameter settings.

Figures (12)

• Figure 1: Illustration of the federated data grouping method. Each client locally slices its dataset into multiple pieces, and the central server randomly shuffles and assigns these slices into balanced groups for sequential group-based training.
• Figure 2: (a) FedSGT overview. Each phase incrementally trains a new PEFT module on the cumulative data groups $\{G_1,\dots,G_i\}$, while freezing previous modules. This design enables exact unlearning by isolating the contribution of each group. (b) Unlearn Strategy: If the earliest trained group (e.g., $G_1$) is removed, all subsequent modules that depend on it become invalid, motivating the need to diversify training orders. (c) Permutation Tree: To improve service maintenance, multiple group permutations are trained as independent sequences, distributing dependencies across different paths so that unlearning a single group only disables a subset of sequences while keeping others functional.
• Figure 3: Counting cyclic span. (a) If the affected groups are 1 and 3, the cyclic span is 3 because the minimum clusters to be removed are group 1, 2, and 3; (b) If the affected groups are 1 and 5, the cyclic span is also 3 because the minimum groups to be removed are 5, 0, and 1.
• Figure 4: Simulation results of performance analysis
• Figure 5: Training performance comparison between FedSGT, FedAvg, and FedCIO under IID and Non-IID settings.

Theorems & Definitions (3)

• Definition 1: Deletion Rate chowdhury2024towards
• proof
• proof