EraseAnything++: Enabling Concept Erasure in Rectified Flow Transformers Leveraging Multi-Object Optimization

TL;DR

Extensive experiments demonstrate that EraseAnything++ substantially outperforms prior methods in erasure effectiveness, generative fidelity, and temporal consistency, establishing a new state of the art for concept erasure in next-generation diffusion models.

Abstract

Removing undesired concepts from large-scale text-to-image (T2I) and text-to-video (T2V) diffusion models while preserving overall generative quality remains a major challenge, particularly as modern models such as Stable Diffusion v3, Flux, and OpenSora employ flow-matching and transformer-based architectures and extend to long-horizon video generation. Existing concept erasure methods, designed for earlier T2I/T2V models, often fail to generalize to these paradigms. To address this issue, we propose EraseAnything++, a unified framework for concept erasure in both image and video diffusion models with flow-matching objectives. Central to our approach is formulating concept erasure as a constrained multi-objective optimization problem that explicitly balances concept removal with preservation of generative utility. To solve the resulting conflicting objectives, we introduce an efficient utility-preserving unlearning strategy based on implicit gradient surgery. Furthermore, by integrating LoRA-based parameter tuning with attention-level regularization, our method anchors erasure on key visual representations and propagates it consistently across spatial and temporal dimensions. In the video setting, we further enhance consistency through an anchor-and-propagate mechanism that initializes erasure on reference frames and enforces it throughout subsequent transformer layers, thereby mitigating temporal drift. Extensive experiments on both image and video benchmarks demonstrate that EraseAnything++ substantially outperforms prior methods in erasure effectiveness, generative fidelity, and temporal consistency, establishing a new state of the art for concept erasure in next-generation diffusion models.

Figures (12)

• Figure 1: Illustration of Difference between EraseAnything++ and Existing Methods. In this paper, we introduce EraseAnything++, an advanced concept erasure technique for Flow-based T2I/T2V images. First row: Classical concept-erasing methods—ESD esd, CA ca, and EAP eap—have been transplanted into Flux.1 [dev] and are tested with the input "$\mathtt{nudity}$" (blue bars indicate author-added sensory harmony). Second row: Visualizing EraseAnything++'s impact—pre and post-concept removal. Original outputs (yellow box) are displayed in the upper right.
• Figure 2: Geometric interpretation of the proposed gradient surgery.(Left) When the erasure gradient $\nabla \mathcal{L}_e$ lies within the safe region (non-conflicting or satisfying the tolerance), the update direction $\boldsymbol{d}_t^*$ remains unchanged. (Right) When a significant conflict arises, $\boldsymbol{d}_t^*$ is obtained by projecting $\nabla \mathcal{L}_e$ to satisfy the preservation constraint. The tolerance $\varepsilon_t$ creates a "tolerance cone" (dashed yellow line) that relaxes strict orthogonality, preventing optimization deadlocks while ensuring controlled utility preservation.
• Figure 3: Correlations between text and attention maps in Flux. Despite the lack of explicit cross-attention layers, a linear relationship persists between text embeddings and the unified attention map.
• Figure 4: Limitations of Attention Map erasure. Zeroing out attention columns ($\mathbf{W_{attn}}[:,:,idx_{i}] = 0$) works for simple prompts but fails against adversarial attacks like keyword obfuscation (e.g., "soccerrs").
• Figure 5: Single concept erasure. We compare our model with mainstream concept erasing methods. The Attention Map column shows the simple token localization method from Section \ref{['sec:sec3']} that erases target concept effectively, yet its vulnerable to the minor change of tokens, making it difficult to widely adopt in practical applications.

Theorems & Definitions (21)

• Proposition 3.1: Dual Problem
• Proposition 3.2: Closed-Form Solution
• Remark 3.3
• Theorem 3.4: Approximate $\lambda^*$
• Theorem 3.5: Pareto Optimality
• Remark 3.6
• Theorem 3.7: Pareto Stationary
• proof
• proof
• proof