Generalizable and Adaptive Continual Learning Framework for AI-generated Image Detection

TL;DR

The paper tackles detecting AI-generated images in a landscape where generators evolve rapidly, risking detector obsolescence. It introduces a three-stage domain continual learning framework: Stage 1 uses parameter-efficient LoRA fine-tuning on CLIP-ViT to build a strong generalizable offline detector; Stage 2 adds data augmentation and K-FAC Hessian-based stabilization to learn from limited new samples without catastrophic forgetting; Stage 3 leverages Linear Mode Connectivity with linear interpolation to exploit shared cross-model features and balance plasticity and stability, formalized as $\theta^* = \frac{1}{T} \sum_{t=1}^T \theta_t$ in the optimal solution. A 27-model, chronologically ordered benchmark across GANs, deepfakes, and diffusion models demonstrates the offline detector's strong generalization plus superior continual-learning performance (AA) of 92.20% and improved robustness to post-processing. The framework shows practical impact for maintaining effective AI-generated image detection in real-world, rapidly changing environments.

Abstract

The malicious misuse and widespread dissemination of AI-generated images pose a significant threat to the authenticity of online information. Current detection methods often struggle to generalize to unseen generative models, and the rapid evolution of generative techniques continuously exacerbates this challenge. Without adaptability, detection models risk becoming ineffective in real-world applications. To address this critical issue, we propose a novel three-stage domain continual learning framework designed for continuous adaptation to evolving generative models. In the first stage, we employ a strategic parameter-efficient fine-tuning approach to develop a transferable offline detection model with strong generalization capabilities. Building upon this foundation, the second stage integrates unseen data streams into a continual learning process. To efficiently learn from limited samples of novel generated models and mitigate overfitting, we design a data augmentation chain with progressively increasing complexity. Furthermore, we leverage the Kronecker-Factored Approximate Curvature (K-FAC) method to approximate the Hessian and alleviate catastrophic forgetting. Finally, the third stage utilizes a linear interpolation strategy based on Linear Mode Connectivity, effectively capturing commonalities across diverse generative models and further enhancing overall performance. We establish a comprehensive benchmark of 27 generative models, including GANs, deepfakes, and diffusion models, chronologically structured up to August 2024 to simulate real-world scenarios. Extensive experiments demonstrate that our initial offline detectors surpass the leading baseline by +5.51% in terms of mean average precision. Our continual learning strategy achieves an average accuracy of 92.20%, outperforming state-of-the-art methods.

Figures (6)

• Figure 1: Testing Accuracy Curves. Accuracy curves for four continual learning baselines (Seq, EWCkirkpatrick2017overcoming, A-GEMchaudhry2018efficient, and SIzenke2017continual) showing the effects of linear interpolation between two adjacent minima on historical tasks (Ap), current tasks (An), and a combined dataset of all tasks (All). The parameter $\lambda$ represents the interpolation factor.
• Figure 2: Framework Overview. A three-stage continual learning framework for AI-generated image detection is presented. Specifically, the first stage employs parameter-efficient fine-tuning strategies using LoRA on MLP layers to develop a generalizable offline detector. The second stage integrates unseen data streams into the continual learning process, proposing a data augmentation chain and leveraging the K-FAC method to acquire new knowledge and prevent catastrophic forgetting. The final stage utilizes linear interpolation to discern and exploit commonalities across different generative models, aiming to balance mode plasticity and stability.
• Figure 3: Task-wise Continual Learning Performance. The heatmap displays the test accuracy (ACC) for each task (x-axis) evaluated at the end of each sequential learning task (y-axis).
• Figure 4: Continual Learning Performance on Proposed Benchmark. We evaluated the performance using ACC and AP metrics across all benchmark datasets after the final round of continual learning. The red dashed line represents the threshold of 50% for both ACC and AP.
• Figure 5: Robustness to Post-processing Operations. We compared the robustness of our method with CNNSpotwang2020cnn and UnivFDojha2023towards against two test-time perturbations: (a) Gaussian Blur and (b) JPEG compression. 'Original' denotes the results obtained from the original images without any post-processing operations.