Machine Perception-Driven Image Compression: A Layered Generative Approach

TL;DR

This work tackles the joint problem of visual data compression and machine perception by introducing a layered generative compression model that operates on two latent streams to deliver reconstruction-oriented and semantics-oriented information. A task-agnostic multi-task analysis network is trained to perform downstream perception directly on compressed representations, enabling substantial bit-rate savings while preserving perceptual and semantic fidelity. The method balances rate, distortion, and perception via a unified optimization framework, with separate and joint training regimes and an AdaIN-based fusion decoder to maximize visual quality and task performance. Experiments on CelebAMask-HQ demonstrate superior perceptual metrics at extreme bitrates and comparable or better machine-perception performance with up to $99.6\%$ bit-rate savings when analyzing compressed-domain features, underscoring practical benefits for scalable vision systems.

Abstract

In this age of information, images are a critical medium for storing and transmitting information. With the rapid growth of image data amount, visual compression and visual data perception are two important research topics attracting a lot attention. However, those two topics are rarely discussed together and follow separate research path. Due to the compact compressed domain representation offered by learning-based image compression methods, there exists possibility to have one stream targeting both efficient data storage and compression, and machine perception tasks. In this paper, we propose a layered generative image compression model achieving high human vision-oriented image reconstructed quality, even at extreme compression ratios. To obtain analysis efficiency and flexibility, a task-agnostic learning-based compression model is proposed, which effectively supports various compressed domain-based analytical tasks while reserves outstanding reconstructed perceptual quality, compared with traditional and learning-based codecs. In addition, joint optimization schedule is adopted to acquire best balance point among compression ratio, reconstructed image quality, and downstream perception performance. Experimental results verify that our proposed compressed domain-based multi-task analysis method can achieve comparable analysis results against the RGB image-based methods with up to 99.6% bit rate saving (i.e., compared with taking original RGB image as the analysis model input). The practical ability of our model is further justified from model size and information fidelity aspects.

Figures (14)

• Figure 1: Image compression framework.
• Figure 2: The framework of conducting machine perception on the reconstructed image.
• Figure 3: The framework of directly conducting machine perception on the compressed visual data.
• Figure 4: The framework of side information assisted layered image compression.
• Figure 5: Overview of our proposed approach. We transform the original image signal into two-layered representations: reconstruction-oriented $I_R$ and semantic-oriented $I_S$. Two targets are illustrated, including 1) compression; 2) machine perception of the compressed representation.