Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Deepfake Detective: Interpreting Neural Forensics Through Sparse Features and Manifolds

Subramanyam Sahoo, Jared Junkin

TL;DR

This work tackles the opacity of deepfake detectors by introducing a mechanistic interpretability framework that couples sparse autoencoder analysis with forensic artifact manifold analysis on a large vision-language backbone. By mapping how learned features respond to controlled artifacts, the authors reveal sparse, artifact-specific latent axes in mid-level layers and a culminating compression of these cues into the final decision. The approach yields quantitative metrics—intrinsic dimensionality, curvature, and feature selectivity—that characterize how different artifacts are represented across network layers, highlighting early-layer sensitivity and providing a roadmap for more interpretable and robust detectors. The methodology, demonstrated on a Qwen2-VL-2B backbone with a 250 real/250 fake subset, offers a generalizable pipeline for diagnosing and improving the interpretability of deepfake detection systems.

Abstract

Deepfake detection models have achieved high accuracy in identifying synthetic media, but their decision processes remain largely opaque. In this paper we present a mechanistic interpretability framework for deepfake detection applied to a vision-language model. Our approach combines a sparse autoencoder (SAE) analysis of internal network representations with a novel forensic manifold analysis that probes how the model's features respond to controlled forensic artifact manipulations. We demonstrate that only a small fraction of latent features are actively used in each layer, and that the geometric properties of the model's feature manifold, including intrinsic dimensionality, curvature, and feature selectivity, vary systematically with different types of deepfake artifacts. These insights provide a first step toward opening the "black box" of deepfake detectors, allowing us to identify which learned features correspond to specific forensic artifacts and to guide the development of more interpretable and robust models.

The Deepfake Detective: Interpreting Neural Forensics Through Sparse Features and Manifolds

TL;DR

This work tackles the opacity of deepfake detectors by introducing a mechanistic interpretability framework that couples sparse autoencoder analysis with forensic artifact manifold analysis on a large vision-language backbone. By mapping how learned features respond to controlled artifacts, the authors reveal sparse, artifact-specific latent axes in mid-level layers and a culminating compression of these cues into the final decision. The approach yields quantitative metrics—intrinsic dimensionality, curvature, and feature selectivity—that characterize how different artifacts are represented across network layers, highlighting early-layer sensitivity and providing a roadmap for more interpretable and robust detectors. The methodology, demonstrated on a Qwen2-VL-2B backbone with a 250 real/250 fake subset, offers a generalizable pipeline for diagnosing and improving the interpretability of deepfake detection systems.

Abstract

Deepfake detection models have achieved high accuracy in identifying synthetic media, but their decision processes remain largely opaque. In this paper we present a mechanistic interpretability framework for deepfake detection applied to a vision-language model. Our approach combines a sparse autoencoder (SAE) analysis of internal network representations with a novel forensic manifold analysis that probes how the model's features respond to controlled forensic artifact manipulations. We demonstrate that only a small fraction of latent features are actively used in each layer, and that the geometric properties of the model's feature manifold, including intrinsic dimensionality, curvature, and feature selectivity, vary systematically with different types of deepfake artifacts. These insights provide a first step toward opening the "black box" of deepfake detectors, allowing us to identify which learned features correspond to specific forensic artifacts and to guide the development of more interpretable and robust models.
Paper Structure (17 sections, 4 equations, 5 figures, 3 tables)

This paper contains 17 sections, 4 equations, 5 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. By integrating these two components, our work provides a deep interpretative insight into which aspects of an image the model uses to detect deepfakes and how those aspects manifest in the model’s internal feature geometry. To our knowledge, this is the first study to decipher a large deepfake detector’s mechanism in terms of its learned feature manifold. In the following, we review related work, then detail our methodology, experimental setup, and findings (excluding quantitative performance results, as our focus is on interpretability).
  4. Related Work
  5. Methodology
  6. Model Architecture and Data Setup
  7. Forensic Artifact Manifold Analysis
  8. Experiments and Analysis
  9. Limitation and Future Works
  10. Conclusion
  11. Enhanced Forensic Analysis Results
  12. Experiment Configuration
  13. Stage 1: Layerwise Forensic Importance Scores
  14. Stage 2: Sparse Autoencoder Statistics
  15. Stage 2b: Artifact Manifold Enumeration
  16. ...and 2 more sections

Figures (5)

  • Figure 1: Forensic importance scores across transformer layers. Lower layers (blocks 0--6) exhibit highest sensitivity to forensic perturbations.
  • Figure 2: Training diagnostics of the enhanced SAE. Top: total loss and component breakdown; Bottom: feature sparsity trends and reconstruction-sparsity balance.
  • Figure 3: Selectivity profile of SAE latent features. Top-left: global distribution; Top-right: Top-50 ranked; Bottom-left: full sorted index; Bottom-right: CDF of absolute selectivity.
  • Figure 4: Causal steering of latent features. Accuracy increases with positive steering magnitude $\alpha$, confirming the directionality and influence of select latent units.
  • Figure 5: Distribution of forensic manifold metrics across artifact types. Includes intrinsic dimensionality (top-left), curvature (top-right), and feature selectivity (bottom-left), with summary statistics (bottom-right).