Why Compress What You Can Generate? When GPT-4o Generation Ushers in Image Compression Fields

TL;DR

This work investigates compressing images by relying on GPT-4o to generate pixel content from compact conditioning signals, rather than transmitting pixel data. It advances two paradigms—textual coding and text+image multimodal coding—by introducing a structure raster-scan prompt to preserve layout and multi-level visual consistency during reconstruction. Experiments on DIV2K at ultra-low bitrates show competitive performance against recent generative codecs without any additional training, highlighting GPT-4o's potential as a practical generative coder. The approach demonstrates that AIGC foundation models can significantly impact image compression by reducing bitrate while maintaining perceptual quality and semantic fidelity, with achievements down to $0.001$ bpp.

Abstract

The rapid development of AIGC foundation models has revolutionized the paradigm of image compression, which paves the way for the abandonment of most pixel-level transform and coding, compelling us to ask: why compress what you can generate if the AIGC foundation model is powerful enough to faithfully generate intricate structure and fine-grained details from nothing more than some compact descriptors, i.e., texts, or cues. Fortunately, recent GPT-4o image generation of OpenAI has achieved impressive cross-modality generation, editing, and design capabilities, which motivates us to answer the above question by exploring its potential in image compression fields. In this work, we investigate two typical compression paradigms: textual coding and multimodal coding (i.e., text + extremely low-resolution image), where all/most pixel-level information is generated instead of compressing via the advanced GPT-4o image generation function. The essential challenge lies in how to maintain semantic and structure consistency during the decoding process. To overcome this, we propose a structure raster-scan prompt engineering mechanism to transform the image into textual space, which is compressed as the condition of GPT-4o image generation. Extensive experiments have shown that the combination of our designed structural raster-scan prompts and GPT-4o's image generation function achieved the impressive performance compared with recent multimodal/generative image compression at ultra-low bitrate, further indicating the potential of AIGC generation in image compression fields.

Figures (9)

• Figure 1: Overall pipeline of our multimodal image compression framework based on GPT-4o image generation.
• Figure 2: Our proposed structural raster-scan prompt.
• Figure 3: Tradeoffs between bitrate and different metrics on DIV2K. The quality is evaluated by both perceptual (blue frames) and consistency metrics (red frames).
• Figure 4: Ablation on prompt length.
• Figure 5: The comparison of qualitative results between the baselines and ours (text+image) method is provided, with the bits per pixel (bpp) for each image listed below. The rate multiples comparing other methods with ours are provided in parentheses.