Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detection

TL;DR

The paper addresses the generalization gap in AI-generated image detection by identifying an asymmetry phenomenon where naive detectors overfit to seen fake patterns, constraining the feature space. It introduces Effort, a SVD-based orthogonal subspace decomposition that freezes principal components to preserve pre-trained vision foundation model representations while learning forgery in a residual subspace. Empirical results on deepfake and synthetic-image benchmarks demonstrate strong generalization with minimal trainable parameters, and analyses reveal a hierarchical prior that fake images derive from real ones, guiding discrimination in semantically aligned subspaces. The approach preserves high-level semantic knowledge while enabling targeted forgery detection, offering a parameter-efficient and broadly applicable solution for robust AIGI detection across diverse domains.

Abstract

AI-generated images (AIGIs), such as natural or face images, have become increasingly important yet challenging. In this paper, we start from a new perspective to excavate the reason behind the failure generalization in AIGI detection, named the \textit{asymmetry phenomenon}, where a naively trained detector tends to favor overfitting to the limited and monotonous fake patterns, causing the feature space to become highly constrained and low-ranked, which is proved seriously limiting the expressivity and generalization. One potential remedy is incorporating the pre-trained knowledge within the vision foundation models (higher-ranked) to expand the feature space, alleviating the model's overfitting to fake. To this end, we employ Singular Value Decomposition (SVD) to decompose the original feature space into \textit{two orthogonal subspaces}. By freezing the principal components and adapting only the remained components, we preserve the pre-trained knowledge while learning fake patterns. Compared to existing full-parameters and LoRA-based tuning methods, we explicitly ensure orthogonality, enabling the higher rank of the whole feature space, effectively minimizing overfitting and enhancing generalization. We finally identify a crucial insight: our method implicitly learns \textit{a vital prior that fakes are actually derived from the real}, indicating a hierarchical relationship rather than independence. Modeling this prior, we believe, is essential for achieving superior generalization. Our codes are publicly available at \href{https://github.com/YZY-stack/Effort-AIGI-Detection}{GitHub}.

Figures (14)

• Figure 1: Illustration of the asymmetry phenomenon in AI-generated image detection. We show that the baseline detector (i.e., Xception) tends to quickly overfit to the fake patterns in the training set rossler2019faceforensics++, causing the limited generalization when facing previously unseen fakes li2019celeb.
• Figure 2: t-SNE visualizations between Xception and CLIP (full-finetune). We show that both models only learn the specific fake patterns within the training set, treating samples with seen fake patterns as fake while other samples are all considered real, thereby limiting their generalization in detecting unseen fakes.
• Figure 3: Analysis for the effective information contained in the model's feature space. We apply PCA for dimension reduction and visualize the explained variance ratio of principal components with high contribution. We show that the baseline model trained on the AIGI dataset can be highly constrained and low-ranked.
• Figure 4: t-SNE visualizations of the latent feature spaces between vanilla CNN wang2020cnn and ours. Our method achieves orthogonal learning between the dimensions of semantic and forgery, allowing the model to capture fake patterns on the semantically-aligned subspace, simplifying the discrimination and thereby improving the generalization.
• Figure 5: The proposed approach for AIGI detection. The left branch is the decomposition matrix of the principle components approximation using SVD, while the right residual branch enables the orthogonal learning of real/fake discriminative features.