Forgetting Through Transforming: Enabling Federated Unlearning via Class-Aware Representation Transformation

TL;DR

Analysis of the representation space reveals FUCRT's ability to effectively merge unlearning class representations with the transformation class from remaining classes, closely mimicking the model retrained from scratch.

Abstract

Federated Unlearning (FU) enables clients to selectively remove the influence of specific data from a trained federated learning model, addressing privacy concerns and regulatory requirements. However, existing FU methods often struggle to balance effective erasure with model utility preservation, especially for class-level unlearning in non-IID settings. We propose Federated Unlearning via Class-aware Representation Transformation (FUCRT), a novel method that achieves unlearning through class-aware representation transformation. FUCRT employs two key components: (1) a transformation class selection strategy to identify optimal forgetting directions, and (2) a transformation alignment technique using dual class-aware contrastive learning to ensure consistent transformations across clients. Extensive experiments on four datasets demonstrate FUCRT's superior performance in terms of erasure guarantee, model utility preservation, and efficiency. FUCRT achieves complete (100\%) erasure of unlearning classes while maintaining or improving performance on remaining classes, outperforming state-of-the-art baselines across both IID and Non-IID settings. Analysis of the representation space reveals FUCRT's ability to effectively merge unlearning class representations with the transformation class from remaining classes, closely mimicking the model retrained from scratch.

Figures (10)

• Figure 1: Distribution of different classes in the model's representation space for the CIFAR10 dataset. The class “automobile” shown in orange, represents the unlearning class. It can be observed that: (1) The representations of the remaining class data exhibit compact and separable clustering characteristics in the representation space of both the original model and the model retrained from scratch. (2) The representations of the unlearning class data (i.e., “automobile”) are concentrated within specific representation domains associated with certain classes (i.e., “truck” and “airplane”) in the representation space of the model retrained from scratch.
• Figure 2: Framework of the proposed FUCRT for unlearning data associated with specific target classes by class-aware representation transformation.
• Figure 3: Test accuracy of remaining class data over training rounds. (Federated unlearning the 10% categories of CIFAR10, CIFAR100, FMNIST, and EuroSAT datasets from pre-trained models.)
• Figure 4: Comparison between TCS and a random label strategies. (The random label strategy maps an unlearning class to a random class. The unlearning class is ‘automobile’.)
• Figure 5: Privacy guarantee of different unlearning methods. (The green line denotes the ASR of retraining from scratch.)