JPEG AI Image Compression Visual Artifacts: Detection Methods and Dataset

TL;DR

This work proposes methods to separately detect three types of artifacts, to localize the affected regions, and to quantify the artifact strength, and considers only those regions that exhibit distortion due solely to the neural compression but that a traditional codec recovers successfully at a comparable bitrate.

Abstract

Learning-based image compression methods have improved in recent years and started to outperform traditional codecs. However, neural-network approaches can unexpectedly introduce visual artifacts in some images. We therefore propose methods to separately detect three types of artifacts (texture and boundary degradation, color change, and text corruption), to localize the affected regions, and to quantify the artifact strength. We consider only those regions that exhibit distortion due solely to the neural compression but that a traditional codec recovers successfully at a comparable bitrate. We employed our methods to collect artifacts for the JPEG AI verification model with respect to HM-18.0, the H.265 reference software. We processed about 350,000 unique images from the Open Images dataset using different compression-quality parameters; the result is a dataset of 46,440 artifacts validated through crowd-sourced subjective assessment. Our proposed dataset and methods are valuable for testing neural-network-based image codecs, identifying bugs in these codecs, and enhancing their performance. We make source code of the methods and the dataset publicly available.

Figures (7)

• Figure 1: Texture artifact found by method \ref{['sec:texture-1']}. JPEG AI replaces most of the holes with lines going in different directions, whereas VTM correctly restores most of these holes. Compression ratio is relative to the original image.
• Figure 2: Texture artifact found by method \ref{['sec:texture-1']}. JPEG AI blurs the texture considerably and reduces detail compared with VTM. Compression ratio is relative to the original image.
• Figure 3: Boundary artifact found by method \ref{['sec:texture-2']}. The edges of the car grill in the JPEG AI-compressed image changed direction compared with the original, whereas VTM avoids this distortion. Compression ratio is relative to the original image.
• Figure 4: Color artifact found by method \ref{['sec:color-1']}. The wheel's hue differs between the JPEG AI result and the original image, whereas VTM restores it correctly. Compression ratio is relative to the original image.
• Figure 5: Color artifact found by method \ref{['sec:color-2']}. JPEG AI failed to restore the red nail color and darkened the cloth, whereas VTM restored them faithfully. Compression ratio is relative to the original image.