SLIM: Semantic-based Low-bitrate Image compression for Machines by leveraging diffusion

TL;DR

SLIM tackles the challenge of low-bitrate image compression for machines by concentrating fidelity on Regions-of-Interest (RoI) and leveraging a pretrained latent diffusion model to enhance reconstructions via RoI-focused captions. The method employs a two-stage training regime to align compression with RoI semantics and downstream classification, combining RoI-guided fidelity, diffusion-based denoising, and task-oriented objectives. Across ImageNet-val with ResNet50 and ConvNeXt-T, SLIM delivers superior rate-accuracy and favorable BD-Rate gains compared with baselines, while qualitative results show natural textures in RoI areas. The work analyzes computational trade-offs inherent to diffusion-based decoding and discusses extensions to larger datasets and faster diffusion, highlighting a practical path toward machine-centric image compression at very low bitrates.

Abstract

In recent years, the demand of image compression models for machine vision has increased dramatically. However, the training frameworks of image compression still focus on the vision of human, maintaining the excessive perceptual details, thus have limitations in optimally reducing the bits per pixel in the case of performing machine vision tasks. In this paper, we propose Semantic-based Low-bitrate Image compression for Machines by leveraging diffusion, termed SLIM. This is a new effective training framework of image compression for machine vision, using a pretrained latent diffusion model.The compressor model of our method focuses only on the Region-of-Interest (RoI) areas for machine vision in the image latent, to compress it compactly. Then the pretrained Unet model enhances the decompressed latent, utilizing a RoI-focused text caption which containing semantic information of the image. Therefore, SLIM is able to focus on RoI areas of the image without any guide mask at the inference stage, achieving low bitrate when compressing. And SLIM is also able to enhance a decompressed latent by denoising steps, so the final reconstructed image from the enhanced latent can be optimized for the machine vision task while still containing perceptual details for human vision. Experimental results show that SLIM achieves a higher classification accuracy in the same bits per pixel condition, compared to conventional image compression models for machines.Code will be released upon acceptance.

Figures (11)

• Figure 1: Architecture of SLIM. The weights of the fixed components are taken from Stable Diffusion.
• Figure 2: Procedure of two-stage training.
• Figure 3: Rate-accuracy performance comparison results. We evaluate classification accuracy on the ImageNet-val dataset by passing reconstructed images from each learned image compression model through machine vision classifiers, ResNet50 and ConvNeXt-T.
• Figure 4: Reconstructed samples from different models, AdaptICMH adapticmh, TransTIC transtic, MPA mpa, and Ours.
• Figure 5: Comparison between SLIM models with and without the diffusion module.