ReLAPSe: Reinforcement-Learning-trained Adversarial Prompt Search for Erased concepts in unlearned diffusion models

TL;DR

ReLAPSe tackles latent leakage after unlearning in diffusion models by reframing restoration as a reinforcement-learning task. It introduces Reinforcement Learning with Verifiable Rewards (RLVR) and Group Relative Policy Optimization (GRPO) to train a prompt-generating policy that aligns text prompts with latent residuals via the diffusion model's noise-prediction loss, yielding transferable strategies. The approach offers both single-target refinement (ReLAPSe-S) and a global, multi-target policy (ReLAPSe-M) that enable scalable adversarial red-teaming across unlearning methods and concepts, with demonstrated improvements in attack success rates and qualitative content recovery. The work provides a scalable diagnostic tool for evaluating unlearning robustness and motivates the development of stronger erasure techniques, with code available at GitHub.

Abstract

Machine unlearning is a key defense mechanism for removing unauthorized concepts from text-to-image diffusion models, yet recent evidence shows that latent visual information often persists after unlearning. Existing adversarial approaches for exploiting this leakage are constrained by fundamental limitations: optimization-based methods are computationally expensive due to per-instance iterative search. At the same time, reasoning-based and heuristic techniques lack direct feedback from the target model's latent visual representations. To address these challenges, we introduce ReLAPSe, a policy-based adversarial framework that reformulates concept restoration as a reinforcement learning problem. ReLAPSe trains an agent using Reinforcement Learning with Verifiable Rewards (RLVR), leveraging the diffusion model's noise prediction loss as a model-intrinsic and verifiable feedback signal. This closed-loop design directly aligns textual prompt manipulation with latent visual residuals, enabling the agent to learn transferable restoration strategies rather than optimizing isolated prompts. By pioneering the shift from per-instance optimization to global policy learning, ReLAPSe achieves efficient, near-real-time recovery of fine-grained identities and styles across multiple state-of-the-art unlearning methods, providing a scalable tool for rigorous red-teaming of unlearned diffusion models. Some experimental evaluations involve sensitive visual concepts, such as nudity. Code is available at https://github.com/gmum/ReLaPSe

Figures (7)

• Figure 1: Methodology overview and qualitative analysis of ReLAPSe. Left: Conceptual overview of the ReLAPSe framework, which utilizes Reinforcement Learning with Verifiable Rewards (RLVR) to discover adversarial prompts for erased concepts. Right: Side-by-side qualitative comparison between original prompts and those generated by our method, demonstrating successful concept recovery across the Nudity, Object, and Style categories.
• Figure 2: Overview of our prompt optimization framework. A frozen, unlearned text-to-image diffusion model is probed by an LLM that generates candidate prompts. For each prompt, we measure the improvement in noise prediction accuracy relative to an unconditional baseline across multiple diffusion timesteps. The LLM is optimized using Group Relative Policy Optimization (GRPO) to amplify prompts that most effectively recover suppressed conditional behavior.
• Figure 3: Qualitative comparison of nudity reconstruction across different methods. See Appendix \ref{['appendix:prompts']} for full generation prompts.
• Figure 4: Examples of Van Gogh's style reconstruction across different methods under ESD and FMN settings. See Appendix \ref{['appendix:prompts']} for full generation prompts.
• Figure 5: Examples of object reconstruction across different methods under ESD setting. The vertical layout allows for clear comparison of visual fidelity and prompt alignment across the benchmarks. See Appendix \ref{['appendix:prompts']} for full generation prompts.