GrOCE:Graph-Guided Online Concept Erasure for Text-to-Image Diffusion Models

TL;DR

GrOCE tackles the challenge of precise, adaptable concept erasure in text-to-image diffusion models by reframing erasure as graph-guided inference in a dynamic semantic space. It introduces three components—Dynamic Topological Graph Construction, Adaptive Cluster Identification, and Selective Edge Severing—to identify and remove entire concept clusters without retraining. The method achieves state-of-the-art results on $CS$ and $FID$ across single, multi-target, and art-style erasure tasks, with real-time performance on a single NVIDIA $A100$ GPU. This graph-based, training-free approach provides interpretable and scalable safety tooling for evolving content risks and copyright concerns in diffusion-based generation.

Abstract

Concept erasure aims to remove harmful, inappropriate, or copyrighted content from text-to-image diffusion models while preserving non-target semantics. However, existing methods either rely on costly fine-tuning or apply coarse semantic separation, often degrading unrelated concepts and lacking adaptability to evolving concept sets. To alleviate this issue, we propose Graph-Guided Online Concept Erasure (GrOCE), a training-free framework that performs precise and adaptive concept removal through graph-based semantic reasoning. GrOCE models concepts and their interrelations as a dynamic semantic graph, enabling principled reasoning over dependencies and fine-grained isolation of undesired content. It comprises three components: (1) Dynamic Topological Graph Construction for incremental graph building, (2) Adaptive Cluster Identification for multi-hop traversal with similarity-decay scoring, and (3) Selective Edge Severing for targeted edge removal while preserving global semantics. Extensive experiments demonstrate that GrOCE achieves state-of-the-art performance on Concept Similarity (CS) and Fréchet Inception Distance (FID) metrics, offering efficient, accurate, and stable concept erasure without retraining.

Figures (6)

• Figure 1: Two key aspects of concept erasure. (a) Concept erasure in text-to-image diffusion models involves both explicit and implicit semantic structure in the latent space. Our method leverages adjacency in semantic space to suppress a target concept while better preserving its neighboring, non-target concepts. (b) Runtime comparison with the training-based ConAbl kumari2023ablating and the recent training-free AdaVD wang2025precise. Our method achieves an order-of-magnitude speedup, making online large-scale concept removal practical.
• Figure 2: The GrOCE pipeline for online concept erasure. Given a user prompt and a specified target concept (e.g., “bear”), GrOCE performs inference-time concept erasure through three core modules: (1) Dynamic Topological Graph Construction builds a semantic graph with vocabulary tokens as nodes and cosine-weighted edges, supporting incremental updates for evolving concept sets.(2)Adaptive Cluster Identification performs multi-hop traversal with similarity decay to identify semantically entangled concepts (e.g., “grizzly,” “panda”) around the target. (3) Selective Edge Severing removes only edges associated with the identified cluster, editing the prompt embedding before diffusion to suppress target semantics while preserving unrelated content.
• Figure 3: From the visualization results, our method demonstrates excellent erasure and retention capabilities, whether it is erasing Snoopy, Snoopy and Mickey, or Snoopy, Mickey and Spongebob. It can not only accurately accomplish target erasure but also stably retain prior knowledge in the process, thus achieving a balance between effectiveness and information retention.
• Figure 4: Impact of hyper-parameters.
• Figure 5: Regarding Van Gogh-related content, we can not only accurately and efficiently completely erase the generated images, but also retain the ability to generate styles of Picasso and Monet, achieving an excellent balance between targeted removal and retention of key information.