PICD: Versatile Perceptual Image Compression with Diffusion Rendering

TL;DR

PICD addresses the challenge of perceptual screen-content compression by encoding text information losslessly and then rendering it together with a compressed image via diffusion rendering. It introduces three conditioning levels—Domain, Adaptor, and Instance—to merge text and image content into a unified reconstruction, achieving both high text accuracy and strong perceptual quality for screen content, while remaining effective for natural images; its framework is mathematically framed by $H(Z|Y)=0$ and $p(\hat{X})=p(X)$. Through a basic implementation and targeted domain-, adaptor-, and instance-level refinements, PICD demonstrates significant gains over state-of-the-art perceptual codecs in text fidelity and perceptual metrics, with ablations confirming the contribution of glyph conditioning, the hybrid adaptor, and guided diffusion sampling. While decoding speed is a current limitation, the approach offers a practical, diffusion-based pathway to versatile perceptual compression that gracefully handles both screen and natural imagery. Overall, PICD bridges screen-content text fidelity and perceptual image quality in a unified diffusion-rendering pipeline, signaling a robust direction for future perceptual codecs.

Abstract

Recently, perceptual image compression has achieved significant advancements, delivering high visual quality at low bitrates for natural images. However, for screen content, existing methods often produce noticeable artifacts when compressing text. To tackle this challenge, we propose versatile perceptual screen image compression with diffusion rendering (PICD), a codec that works well for both screen and natural images. More specifically, we propose a compression framework that encodes the text and image separately, and renders them into one image using diffusion model. For this diffusion rendering, we integrate conditional information into diffusion models at three distinct levels: 1). Domain level: We fine-tune the base diffusion model using text content prompts with screen content. 2). Adaptor level: We develop an efficient adaptor to control the diffusion model using compressed image and text as input. 3). Instance level: We apply instance-wise guidance to further enhance the decoding process. Empirically, our PICD surpasses existing perceptual codecs in terms of both text accuracy and perceptual quality. Additionally, without text conditions, our approach serves effectively as a perceptual codec for natural images.

Figures (15)

• Figure 1: For both screen and natural images, PICD demonstrates high text accuracy and superior visual quality simultaneously.
• Figure 2: PICD works well for both screen and natural images.
• Figure 3: The overall framework of our proposed PICD.
• Figure 4: An example of the PICD pipeline. PICD first extracts and encodes text information $Z$ from source $X$. Then, PICD converts text information into text glyph $\bar{Z}$. Finally, PICD renders glyph $\bar{Z}$ with a compressed image $\bar{X}$ into reconstruction $\hat{X}$ using diffusion.
• Figure 5: Ablation studies on different components of diffusion rendering.