One Image is Worth a Thousand Words: A Usability Preservable Text-Image Collaborative Erasing Framework

TL;DR

This paper tackles the safety challenge in text-to-image diffusion by addressing the limitations of text-only concept erasing. It introduces Co-Erasing, a text-image collaborative framework that uses visual templates generated from the model itself to supplement text prompts, and a text-guided image refinement module to focus on concept-relevant visual features. The approach achieves a better trade-off between erasure efficacy and general usability, outperforming state-of-the-art baselines across nudity, style, and object erasure tasks, and demonstrating robustness across backbones and multi-concept scenarios. By training with a dual-branch encoder and self-generated images, Co-Erasing bypasses the text–image gap and provides stronger, more reliable concept erasure with minimal impact on benign generation, as validated by comprehensive quantitative and qualitative experiments.

Abstract

Concept erasing has recently emerged as an effective paradigm to prevent text-to-image diffusion models from generating visually undesirable or even harmful content. However, current removal methods heavily rely on manually crafted text prompts, making it challenging to achieve a high erasure (efficacy) while minimizing the impact on other benign concepts (usability). In this paper, we attribute the limitations to the inherent gap between the text and image modalities, which makes it hard to transfer the intricately entangled concept knowledge from text prompts to the image generation process. To address this, we propose a novel solution by directly integrating visual supervision into the erasure process, introducing the first text-image Collaborative Concept Erasing (Co-Erasing) framework. Specifically, Co-Erasing describes the concept jointly by text prompts and the corresponding undesirable images induced by the prompts, and then reduces the generating probability of the target concept through negative guidance. This approach effectively bypasses the knowledge gap between text and image, significantly enhancing erasure efficacy. Additionally, we design a text-guided image concept refinement strategy that directs the model to focus on visual features most relevant to the specified text concept, minimizing disruption to other benign concepts. Finally, comprehensive experiments suggest that Co-Erasing outperforms state-of-the-art erasure approaches significantly with a better trade-off between efficacy and usability. Codes are available at https://github.com/Ferry-Li/Co-Erasing.

Figures (39)

• Figure 1: The original prompt describes a painting of Padme Amidala, a fictional character from the Star War. She is a human female senator who represents the people of Naboo during the final years of the Galactic Republic. Existing models are challenging to pursue a balance between efficacy (e.g., ESD ESD, FMN FMN) and usability (SalUn SalUn, AdvUnlearn AdvUnlearn), while ours can satisfy both requirements simultaneously.
• Figure 2: Performance overview of Co-Erasing and other methods when erasing nudity. A higher ASR indicates a lower erasing efficacy, meaning the target concept is not erased completely. A higher FID or a lower CLIP score indicates a lower general usability, meaning the benign generation is degraded. Competitive methods include FMN FMN, ESD ESD, SPM SPM, UCE UCE, SalUn SalUn, SH SH, ED ED and AdvUnlearn AdvUnlearn.
• Figure 3: Semantically benign words generate inappropriate content. The word "rhodesian" and "birth" activates the explicit part although not related to the concept nudity.
• Figure 4: Visualization and performance of erasing nudity with different numbers of words. Specific words for (b), (c) and (d) are listed in \ref{['app:word_number']}.
• Figure 5: Similarity between failure cases of nudity and (a) text "nudity" and (b) self-generated NSFW images. Both text and images are processed by CLIP before calculating the similarity.