Revealing the Implicit Noise-based Imprint of Generative Models

TL;DR

NIRNet addresses the challenge of detecting AI-generated images with strong generalization to unseen generators by focusing on universal noise-based imprints from the synthesis process. It introduces a Noise-based Imprint Simulator to expand training data through fused imprints from multiple models and a Noise-based Imprint Extractor paired with a hybrid discriminator that combines noise, frequency, and semantic cues. The method achieves state-of-the-art results across seven benchmarks and two new generalization tests (Gen-8K and ForenGen), demonstrating robustness to cross-model variation and in-the-wild post-processing. By shifting detection toward fundamental imprint patterns rather than model-specific artifacts, NIRNet offers practical implications for safeguarding against misinformation and copyright violations in AI-generated imagery.

Abstract

With the rapid advancement of vision generation models, the potential security risks stemming from synthetic visual content have garnered increasing attention, posing significant challenges for AI-generated image detection. Existing methods suffer from inadequate generalization capabilities, resulting in unsatisfactory performance on emerging generative models. To address this issue, this paper presents NIRNet (Noise-based Imprint Revealing Network), a novel framework that leverages noise-based imprint for the detection task. Specifically, we propose a novel Noise-based Imprint Simulator to capture intrinsic patterns imprinted in images generated by different models. By aggregating imprint from various generative models, imprint of future models can be extrapolated to expand training data, thereby enhancing generalization and robustness. Furthermore, we design a new pipeline that pioneers the use of noise patterns, derived from a Noise-based Imprint Extractor, alongside other visual features for AI-generated image detection, significantly improving detection performance. Our approach achieves state-of-the-art performance across seven diverse benchmarks, including five public datasets and two newly proposed generalization tests, demonstrating its superior generalization and effectiveness. Paper Submission: pdf

Figures (6)

• Figure 1: Conventional detectors (Top) perform well on known generative models, but typically struggle with images from unseen architectures. Our approach (Bottom) utilizes a Noise-based Imprint Extractor to capture a universal generative noise patterns, distinct from the real one, ensuring robust generalization.
• Figure 2: Conventional detectors tend to overfit to Known generative models (orange area), leading to poor performance on Unseen models (red dots). Our method, using a Noise-based Imprint Simulator, learns an extrapolated synthetic boundary (grey area) that covers unseen models, thus achieving robust generalization.
• Figure 3: Overall framework of the proposed NIRNet. Our NIRNet consists of two stages: a simulation stage and a training stage. In the simulation stage (a) and (b), a Noise-based Imprint Simulator models the noise-based imprint of the generative model. This process computes differences after reconstruction, which are learned and fitted to a Laplace distribution. Subsequently, samples are drawn from a fused distribution derived from multiple models to transform real images into images embedded with imprint, thereby expanding the training dataset. In the training stage (c), end-to-end training is performed on the expanded dataset. A Noise-based Imprint Extractor, utilized to capture the intrinsic noise patterns of images, is introduced. In conjunction with frequency and semantic features, our framework functions in a hybrid feature manner to detect AI-generated images.
• Figure 4: PCA visualizations of noise-based imprint distributions (a) from four different models, and (b) with the addition of a fused distribution used in Noise-based Imprint Simulator. The fused distribution helps simulate unseen or future model imprint.
• Figure 5: Architecture of the Difference-Aware Loss. This auxiliary loss trains the Noise-based Imprint Extractor to produce noise feature differences that are predictive of the latent representation differences computed by a fixed VAE Encoder. An MLP is trained as a projection module to regress these differences, and the resulting MSE loss guides the extractor to learn pixel-level patterns that correlate with variations in the latent generative space.