Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Architecture-Agnostic Feature Synergy for Universal Defense Against Heterogeneous Generative Threats

Bingxue Zhang, Yang Gao, Feida Zhu, Yanyan Shen, Yang Shi

Abstract

Generative AI deployment poses unprecedented challenges to content safety and privacy. However, existing defense mechanisms are often tailored to specific architectures (e.g., Diffusion Models or GANs), creating fragile "defense silos" that fail against heterogeneous generative threats. This paper identifies a fundamental optimization barrier in naive pixel-space ensemble strategies: due to divergent objective functions, pixel-level gradients from heterogeneous generators become statistically orthogonal, causing destructive interference. To overcome this, we observe that despite disparate low-level mechanisms, high-level feature representations of generated content exhibit alignment across architectures. Based on this, we propose the Architecture-Agnostic Targeted Feature Synergy (ATFS) framework. By introducing a target guidance image, ATFS reformulates multi-model defense as a unified feature space alignment task, enabling intrinsic gradient alignment without complex rectification. Extensive experiments show ATFS achieves SOTA protection in heterogeneous scenarios (e.g., Diffusion+GAN). It converges rapidly, reaching over 90% performance within 40 iterations, and maintains strong attack potency even under tight perturbation budgets. The framework seamlessly extends to unseen architectures (e.g., VQ-VAE) by switching the feature extractor, and demonstrates robust resistance to JPEG compression and scaling. Being computationally efficient and lightweight, ATFS offers a viable pathway to dismantle defense silos and enable universal generative security. Code and models are open-sourced for reproducibility.

Architecture-Agnostic Feature Synergy for Universal Defense Against Heterogeneous Generative Threats

Abstract

Generative AI deployment poses unprecedented challenges to content safety and privacy. However, existing defense mechanisms are often tailored to specific architectures (e.g., Diffusion Models or GANs), creating fragile "defense silos" that fail against heterogeneous generative threats. This paper identifies a fundamental optimization barrier in naive pixel-space ensemble strategies: due to divergent objective functions, pixel-level gradients from heterogeneous generators become statistically orthogonal, causing destructive interference. To overcome this, we observe that despite disparate low-level mechanisms, high-level feature representations of generated content exhibit alignment across architectures. Based on this, we propose the Architecture-Agnostic Targeted Feature Synergy (ATFS) framework. By introducing a target guidance image, ATFS reformulates multi-model defense as a unified feature space alignment task, enabling intrinsic gradient alignment without complex rectification. Extensive experiments show ATFS achieves SOTA protection in heterogeneous scenarios (e.g., Diffusion+GAN). It converges rapidly, reaching over 90% performance within 40 iterations, and maintains strong attack potency even under tight perturbation budgets. The framework seamlessly extends to unseen architectures (e.g., VQ-VAE) by switching the feature extractor, and demonstrates robust resistance to JPEG compression and scaling. Being computationally efficient and lightweight, ATFS offers a viable pathway to dismantle defense silos and enable universal generative security. Code and models are open-sourced for reproducibility.
Paper Structure (30 sections, 6 equations, 10 figures, 3 tables, 1 algorithm)

This paper contains 30 sections, 6 equations, 10 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Architecture-Specific Proactive Defense
  4. Adversarial Transferability and Ensemble Strategies
  5. Multi-Objective Optimization and Gradient Conflict
  6. Method
  7. Motivation and Problem Formulation
  8. Formalization of the Heterogeneous Defense Problem
  9. Analysis of Gradient Conflict in Pixel Space
  10. Heuristic Observation: Feature Space Alignment
  11. Shared Feature Space Alignment Framework
  12. Feature Extraction and Shared Feature Space
  13. Feature Space Synergistic Optimization Mechanism
  14. Working Mechanism of Feature Synergy
  15. Algorithm Implementation and Details
  16. ...and 15 more sections

Figures (10)

  • Figure 1: Defense Silos Phenomenon: Architecture-Specific Attacks Fail Cross-Model. The figure illustrates how perturbations optimized for one model architecture fail to transfer to another. A checkmark ($\checkmark$) indicates successful defense (editing failure), while a cross ($\times$) indicates defense failure (editing success). Facial images protected by diffusion-specific methods remain vulnerable to GAN-based edits, and vice versa.
  • Figure 2: Universal Feature Anchoring: Unifying Heterogeneous Models via Shared Feature Space
  • Figure 3: Qualitative Visualization of Cross-Architecture Defense Effects
  • Figure 4: Transferability to DreamBooth. Adversarial examples generated by ATFS effectively disrupt subject-driven generation in DreamBooth
  • Figure 5: Qualitative Results on DM + VQ-VAE Scenario
  • ...and 5 more figures