FaSDiff: Balancing Perception and Semantics in Face Compression via Stable Diffusion Priors

TL;DR

FaSDiff addresses the trade-off between perceptual fidelity and semantic consistency in facial image compression by using a diffusion-prior guided pipeline. It jointly optimizes a high-frequency aware compressor on latent $y$ and a hybrid low-frequency semantic enhancement to steer the diffusion process with prompts $s_t$ and $s_f$, producing high-quality reconstructions $x_{rec}$. Time-aware high-frequency augmentation (TaHFA) and hybrid low-frequency enhancement (HLFE) decouple frequency channels to stabilize diffusion-guided reconstruction while preserving facial semantics. Across CelebA-HQ and Facescrub, FaSDiff achieves state-of-the-art performance on perceptual metrics and maintains downstream task accuracy at very low bitrates, demonstrating practical impact for storage, transmission, and downstream analytics.

Abstract

With the increasing deployment of facial image data across a wide range of applications, efficient compression tailored to facial semantics has become critical for both storage and transmission. While recent learning-based face image compression methods have achieved promising results, they often suffer from degraded reconstruction quality at low bit rates. Directly applying diffusion-based generative priors to this task leads to suboptimal performance in downstream machine vision tasks, primarily due to poor preservation of high-frequency details. In this work, we propose FaSDiff (\textbf{Fa}cial Image Compression with a \textbf{S}table \textbf{Diff}usion Prior), a novel diffusion-driven compression framework designed to enhance both visual fidelity and semantic consistency. FaSDiff incorporates a high-frequency-sensitive compressor to capture fine-grained details and generate robust visual prompts for guiding the diffusion model. To address low-frequency degradation, we further introduce a hybrid low-frequency enhancement module that disentangles and preserves semantic structures, enabling stable modulation of the diffusion prior during reconstruction. By jointly optimizing perceptual quality and semantic preservation, FaSDiff effectively balances human visual fidelity and machine vision accuracy. Extensive experiments demonstrate that FaSDiff outperforms state-of-the-art methods in both perceptual metrics and downstream task performance.

Figures (9)

• Figure 1: The trade-off between different compression methods for perceptual quality and downstream task performance. Proximity to the upper right corner indicates superior overall model performance. The color indicates the level of compression rate.
• Figure 2: (Left): The basic T2I LDM generation mode struggles to produce controllable outputs with stable details while maintaining strong fidelity in the images. (Right): Overview of our proposed FaSDiff and qualitative comparison with mainstream solutions. FaSDiff employs a blend of low-frequency and high-frequency control, reconstructing intricate details at extremely low bit rates with perfect realism.
• Figure 3: Illustration of the proposed Facial Image Compression with a Stable Diffusion prior (FaSDiff) framework. Initially, we extract $\boldsymbol{y}$ of the input image $\boldsymbol{x}$ through encoder $\mathcal{E}$. Subsequently, guided by facial consistency loss, we employ an end-to-end compressor to obtain the compressed high-frequency image control flow (solid blue line). Simultaneously, we decouple low-frequency facial semantics from the image prompts, generating a hybrid low-frequency semantic control flow (dashed red line). Ultimately, under the guidance of the cross-prompt interaction between high and low frequencies, the diffusion prior generates the final result $\boldsymbol{x}_{rec}$.
• Figure 4: (a)-(c): An example using face consistency guidance. The face consistency loss makes the generated facial expressions more faithful to the original image. (d): Visualization results of the standardized variance $var(\boldsymbol{y})$.
• Figure 5: (a)-(b): Fourier spectrum of W and W/o TaHFA. (c): Relative log amplitudes of Fourier-transformed feature maps $\boldsymbol{z}_{\boldsymbol{y}}$.