Generative Unlearning for Any Identity

TL;DR

GUIDE introduces generative identity unlearning to erase a specified identity from pre-trained GANs using a single image, addressing privacy risks in generative models. It combines Un-Identifying Face On Latent Space (UFO) to select a latent target and Latent Target Unlearning (LTU) with three losses—Local, Adjacency, and Global Preservation—to shift identity while preserving the model's overall distribution. Experiments on 3D EG3D (FFHQ) and 2D StyleGAN2 (FFHQ/CelebAHQ) demonstrate state-of-the-art identity erasure with minimal collateral changes and quantifiable maintenance of generation quality, including unseen identities in OOD settings. The work provides a practical privacy-preserving tool and a framework for identity unlearning in generative models, with broad implications for safe deployment of powerful generative systems.

Abstract

Recent advances in generative models trained on large-scale datasets have made it possible to synthesize high-quality samples across various domains. Moreover, the emergence of strong inversion networks enables not only a reconstruction of real-world images but also the modification of attributes through various editing methods. However, in certain domains related to privacy issues, e.g., human faces, advanced generative models along with strong inversion methods can lead to potential misuses. In this paper, we propose an essential yet under-explored task called generative identity unlearning, which steers the model not to generate an image of a specific identity. In the generative identity unlearning, we target the following objectives: (i) preventing the generation of images with a certain identity, and (ii) preserving the overall quality of the generative model. To satisfy these goals, we propose a novel framework, Generative Unlearning for Any Identity (GUIDE), which prevents the reconstruction of a specific identity by unlearning the generator with only a single image. GUIDE consists of two parts: (i) finding a target point for optimization that un-identifies the source latent code and (ii) novel loss functions that facilitate the unlearning procedure while less affecting the learned distribution. Our extensive experiments demonstrate that our proposed method achieves state-of-the-art performance in the generative machine unlearning task. The code is available at https://github.com/KHU-AGI/GUIDE.

Figures (17)

• Figure 1: Given a single source image containing a specific identity, we remove that identity from the pre-trained 3D generative adversarial network (e.g., EG3D chan2022efficient). Our method effectively unlearns identity even from in-the-wild images where the source image is absent in the pre-training dataset.
• Figure 2: An illustration of generative identity unlearning. Upon GUIDE, the identity of the image generated from $w_u$, i.e., inversion of the source image $x_u$ by inversion network $E$, should exhibit a distinct identity when passed through the pre-trained generator $G_s$ compared to the unlearned generator $G_u$. Furthermore, other images $x_o$, not used in unlearning but sharing the same identity with $x_u$, also should vary an identity through GUIDE.
• Figure 3: An overview of GUIDE. Starting with a source image, we employ a GAN inversion network $E$, specifically GOAE yuan2023make, to embed this image into the latent space of a pre-trained generative model, namely EG3D chan2022efficient, obtaining the source latent code $w_u$. The target latent code $w_t$ is designated through the UFO process. To facilitate identity removal in $w_u$, we shift its identity to match that of $w_t$ with our Latent Target Unlearning (LTU) process. Three loss functions of LTU are designed for this purpose: (i) The generator is optimized to produce an image from the source latent code, denoted as $(R \circ G_u)(w_u)$, that is similar to the image from the target latent code, represented as $(R \circ G_s)(w_t)$. (ii) To achieve unlearning across the entire identity, we consider latent codes near both the source and target latent codes, denoted as $w_{u,a}$ and $w_{t,a}$, respectively. (iii) To prevent model corruption during the unlearning process, we additionally sample latent codes from a random noise vector, represented as $w_{r,g}$, and optimize $G_u$ to preserve its generation ability on $w_{r,g}$.
• Figure 4: An illustration of Un-identifying Face On Latent Space (UFO). We define the identity of the source latent code by subtract it from the average latent code. We set the target latent code for our unlearning process by measuring an extrapolation between the source and average latent code with a fixed distance $d$.
• Figure 5: An illustration of determining latent codes near a latent code $w$ in adjacency-aware unlearning loss. We first sample a latent code $w_{r,a}$ which is derived from a random noise vector $z_{r,a}$ via the mapping network $Map(\cdot)$, i.e.$w_{r,a}=Map(z_{r,a})$. Next, we compute the direction between $w$ and $w_{r,a}$, and we scale it to fall within range between 0 and $\alpha_{max}$. This yields the distance vector $\Delta$ to compute the adjacent latent code $w_a=w+\Delta$.