From Spurious to Causal: Low-rank Orthogonal Subspace Intervention for Generalizable Face Forgery Detection

TL;DR

This work tackles the generalization gap in face forgery detection by modeling spurious correlations as a compressible, low-rank subspace within CLIP's visual representations. It introduces Low-rank Orthogonal Removal (LROR) via SeLop, which learns an orthogonal basis $\mathbf{Q}$ to project out spurious directions $X_{vis}\mathbf{Q}\mathbf{Q}^T$, keeping the orthogonal complement that encodes forgery cues for classification. By only training the projection and a linear head, the method preserves CLIP's pre-trained knowledge while reducing spurious influence, achieving state-of-the-art robustness with approximately 0.43M parameters. The approach is supported by theoretical analysis of representation separability, and extensive experiments across cross-dataset and cross-manipulation benchmarks demonstrate superior generalization and robustness to perturbations, highlighting the practical impact of causal representation interventions in vision tasks. Key contributions include formulating spurious correlations as a whole via a causal framework, developing a lightweight yet effective LROR mechanism, and providing comprehensive ablations and analyses that substantiate the method’s effectiveness.

Abstract

The generalization problem remains a critical challenge in face forgery detection. Some researches have discovered that ``a backdoor path" in the representations from forgery-irrelevant information to labels induces biased learning, thereby hindering the generalization. In this paper, these forgery-irrelevant information are collectively termed spurious correlations factors. Previous methods predominantly focused on identifying concrete, specific spurious correlation and designing corresponding solutions to address them. However, spurious correlations arise from unobservable confounding factors, making it impractical to identify and address each one individually. To address this, we propose an intervention paradigm for representation space. Instead of tracking and blocking various instance-level spurious correlation one by one, we uniformly model them as a low-rank subspace and intervene in them. Specifically, we decompose spurious correlation features into a low-rank subspace via orthogonal low-rank projection, subsequently removing this subspace from the original representation and training its orthogonal complement to capture forgery-related features. This low-rank projection removal effectively eliminates spurious correlation factors, ensuring that classification decision is based on authentic forgery cues. With only 0.43M trainable parameters, our method achieves state-of-the-art performance across several benchmarks, demonstrating excellent robustness and generalization.

Figures (5)

• Figure 1: (a) An example of the impact of spurious correlations on Vanilla CLIP clip. A backdoor path exists from forgery-irrelevant information (e.g. background style, identity information) to labels. (b) Causal intervention. We propose orthogonal low-rank projection to remove the spurious correlation subspace, disrupting the backdoor path.
• Figure 2: Pipeline of the proposed method SeLop. Based on the Structural Causal Model, we propose Low-rank Orthogonal Removal of Spurious Correlation (LROR) to mitigate influence of forgery-irrelevant factors $X_s$, which aims to break the backdoor path from spurious correlations to labels. See text for details.
• Figure 3: Robustness Analysis. Our method is compared with CLIPclip, SRMsrm, and COREcore across five levels of six particular types of perturbations in video-level AUC.
• Figure 4: T-SNE Visualizations. The feature distribution before and after intervention in visual space of CLIPclip: Vanilla CLIP (left, before intervention), spurious correlation subspace (mid), our method (right, after intervention).
• Figure 5: Comparison between our method and other SOTA detectors on the DFDC dataset (trained on FF++). Our method needs the fewest learnable parameters while achieves the best performance.