A Taxonomy of Miscompressions: Preparing Image Forensics for Neural Compression

TL;DR

The problem space is explored and a provisional taxonomy of miscompressions is proposed, which defines three types of “what happens” and has a binary “high impact” flag indicating miscompressions that alter symbols.

Abstract

Neural compression has the potential to revolutionize lossy image compression. Based on generative models, recent schemes achieve unprecedented compression rates at high perceptual quality but compromise semantic fidelity. Details of decompressed images may appear optically flawless but semantically different from the originals, making compression errors difficult or impossible to detect. We explore the problem space and propose a provisional taxonomy of miscompressions. It defines three types of 'what happens' and has a binary 'high impact' flag indicating miscompressions that alter symbols. We discuss how the taxonomy can facilitate risk communication and research into mitigations.

Figures (6)

• Figure 1: State-of-the-art neural compression schemes can alter the semantic in details of the decompressed images. The high fidelity and the lack of visible compression artifacts make false reconstructions look more authentic than JPEG, which introduces visible distortion. (Crop of image $0831$ of DIV2K Agustsson_2017DIV2K, $0.41\%$ of the original.) All figures are best viewed on screen and magnified.
• Figure 2: Comparison of the conventional JPEG compression (top) and the neural compression schemes used in this paper (bottom). Elements with rounded corners are CNNs trained on datasets and used in inference mode during encoding and reconstruction.
• Figure 3: Category Amplitude: Reconstructions differ in the amplitude of spatial frequencies in the signal, affecting attributes such as brightness, color saturation, and the intensity of high frequency components.
• Figure 4: Category Geometry: Reconstructions contain geometric transformations, such as translation, rotation, scaling, and shearing, including shifted shapes and dissolved contours. (The top image illustrates the level of detail at which miscompressions occur. A grid was added to the middle crop.)
• Figure 5: Category Shape: Reconstructions contain changed contours.