Is JPEG AI going to change image forensics?

TL;DR

The paper investigates how JPEG AI, a neural end-to-end image compression standard, introduces counter-forensic effects that challenge current multimedia forensics. By evaluating multiple deepfake detectors and splicing localization methods on a wide range of datasets and six JPEG AI compression settings, it shows that JPEG AI can shift score distributions of pristine content toward synthetic regions and degrade tampering masks, more so than standard JPEG. It also demonstrates that retraining detectors with JPEG AI data can mitigate some effects, though not fully, and that double JPEG AI compression yields complex interactions that depend on compression order and settings. The findings underscore the need to include JPEG AI images in forensic evaluations and to develop robust, artifact-discriminating techniques, potentially complemented by provenance and integrity frameworks for trusted media.

Abstract

In this paper, we investigate the counter-forensic effects of the new JPEG AI standard based on neural image compression, focusing on two critical areas: deepfake image detection and image splicing localization. Neural image compression leverages advanced neural network algorithms to achieve higher compression rates while maintaining image quality. However, it introduces artifacts that closely resemble those generated by image synthesis techniques and image splicing pipelines, complicating the work of researchers when discriminating pristine from manipulated content. We comprehensively analyze JPEG AI's counter-forensic effects through extensive experiments on several state-of-the-art detectors and datasets. Our results demonstrate a reduction in the performance of leading forensic detectors when analyzing content processed through JPEG AI. By exposing the vulnerabilities of the available forensic tools, we aim to raise the urgent need for multimedia forensics researchers to include JPEG AI images in their experimental setups and develop robust forensic techniques to distinguish between neural compression artifacts and actual manipulations.

Figures (20)

• Figure 1: Main findings of our experimental campaign on the counter-forensic effect of JPEG AI. On the top row, we report the deepfake detection scores of pristine images and their JPEG AI-compressed versions as the compression ratio increases (darker red tones), proving that pristine images compressed with this new standard can be mistaken for deepfakes. On the bottom row, we show that, after JPEG AI compression, localization maps produced by state-of-the-art algorithms can no longer distinguish genuine from manipulated content.
• Figure 2: JPEG AI framework.
• Figure 3: Example of JPEG AI artifacts left at different values in the Fourier and spatial domains. Left block: FFHQ dataset; right block: LSUN dataset. In the first row, we show the average Fourier spectra (magnitude) of pristine and JPEG AI images; all spectra are centered in the spatial frequencies $(0, 0)$. In the second row, we show the close-up of one image per dataset. Best viewed in electronic format.
• Figure 4: Scores distribution for gragnaniello2021 over COCO pristine samples (top row) and their synthetic counterpart samples (bottom row) compressed at different values.
• Figure 5: Scores distribution for cozzolino2023raising-A over LSUN pristine samples (top row) and their synthetic counterpart (bottom row) compressed with JPEG at different values.