Verifiably Forgotten? Gradient Differences Still Enable Data Reconstruction in Federated Unlearning

TL;DR

This work reveals a privacy vulnerability in verifiable Federated Unlearning: when PoFU is based on gradient differences, an honest-but-curious auditor can reconstruct forgotten samples. It introduces IGF, a learning-based inversion framework that uses SVD for dimensionality reduction and a pixel-level inversion network trained with a composite loss to map projected gradient differences to forgotten data. The authors validate IGF across multiple datasets, FU scenarios, and model architectures, showing strong reconstruction fidelity and highlighting weaknesses in existing FU defenses. To mitigate the threat, they propose an orthogonal obfuscation defense that preserves PoFU verification while disrupting the directional information critical for reconstruction, underscoring the need for robust FU verification methods and more resilient protections against gradient-based inferences.

Abstract

Federated Unlearning (FU) has emerged as a critical compliance mechanism for data privacy regulations, requiring unlearned clients to provide verifiable Proof of Federated Unlearning (PoFU) to auditors upon data removal requests. However, we uncover a significant privacy vulnerability: when gradient differences are used as PoFU, honest-but-curious auditors may exploit mathematical correlations between gradient differences and forgotten samples to reconstruct the latter. Such reconstruction, if feasible, would face three key challenges: (i) restricted auditor access to client-side data, (ii) limited samples derivable from individual PoFU, and (iii) high-dimensional redundancy in gradient differences. To overcome these challenges, we propose Inverting Gradient difference to Forgotten data (IGF), a novel learning-based reconstruction attack framework that employs Singular Value Decomposition (SVD) for dimensionality reduction and feature extraction. IGF incorporates a tailored pixel-level inversion model optimized via a composite loss that captures both structural and semantic cues. This enables efficient and high-fidelity reconstruction of large-scale samples, surpassing existing methods. To counter this novel attack, we design an orthogonal obfuscation defense that preserves PoFU verification utility while preventing sensitive forgotten data reconstruction. Experiments across multiple datasets validate the effectiveness of the attack and the robustness of the defense. The code is available at https://anonymous.4open.science/r/IGF.

Figures (14)

• Figure 1: Audition in verifiable FU
• Figure 2: Schematic overview of IGF framework. A. Learning Phase: Clients collaboratively train the global model via FL. B. Unlearning Phase: The unlearned clients are required to forget specific data contributions and submit the proof of federated unlearning (PoFU). C. Verification & Attack Phase: The honest-but-curious auditor verifies PoFUs, while attempting to infer forgotten data using a pre-trained inversion model $\mathcal{I}$.
• Figure 3: Schematic of orthogonal obfuscation defense
• Figure 4: Original forgotten images and our reconstructed images on the CIFAR-10 dataset when the number of forgotten samples is 1000.
• Figure 5: Forgotten images and our reconstructed images on the CIFAR-10 dataset under Orthogonal Obfuscation defense.