Switchable Token-Specific Codebook Quantization For Face Image Compression

TL;DR

This work tackles the challenge of face-image compression at ultra-low bitrates by identifying the inadequacy of a single global codebook. It introduces Switchable Token-Specific Codebook Quantization (STSCQ), which combines image-level codebook routing with token-level sub-codebooks to expand capacity while controlling bit usage. The method employs a three-stage training strategy and an entropy-aware routing objective to maintain codebook utilization and reconstruction fidelity, and it demonstrates improved recognition accuracy at low bpp on face datasets. The approach is plug-and-play with existing codebook-based compressors and offers practical gains in rate-distortion performance and identity preservation for compressed facial imagery.

Abstract

With the ever-increasing volume of visual data, the efficient and lossless transmission, along with its subsequent interpretation and understanding, has become a critical bottleneck in modern information systems. The emerged codebook-based solution utilize a globally shared codebook to quantize and dequantize each token, controlling the bpp by adjusting the number of tokens or the codebook size. However, for facial images, which are rich in attributes, such global codebook strategies overlook both the category-specific correlations within images and the semantic differences among tokens, resulting in suboptimal performance, especially at low bpp. Motivated by these observations, we propose a Switchable Token-Specific Codebook Quantization for face image compression, which learns distinct codebook groups for different image categories and assigns an independent codebook to each token. By recording the codebook group to which each token belongs with a small number of bits, our method can reduce the loss incurred when decreasing the size of each codebook group. This enables a larger total number of codebooks under a lower overall bpp, thereby enhancing the expressive capability and improving reconstruction performance. Owing to its generalizable design, our method can be integrated into any existing codebook-based representation learning approach and has demonstrated its effectiveness on face recognition datasets, achieving an average accuracy of 93.51% for reconstructed images at 0.05 bpp.

Figures (7)

• Figure 1: Storage cost comparison between previous latent space models and our method. (a) Previous latent space model: Global-shared codebook requiring storage cost of $T \lceil \log_2 N \rceil$ bits. (b) Our method: Token-specific codebook selection reduces storage to $T \lceil \log_2 K \rceil + \lceil \log_2 M \rceil$ bits.
• Figure 2: (a) Overview of the proposed architecture. (b) Dynamic switching mechanism for token-specific codebook selection. (c) Composition of the $i$-th token-specific codebook. (d) Token-specific quantization and dequantization process for the $j$-th token. Sample face images are from the FFHQ dataset karras2019style.
• Figure 3: Illustration of the training strategy. (a) Training stage 1: Optimization of the switchable token-shared codebook. (b) Training stage 2: Optimization of switchable token-specific codebooks, initialized from the Stage 1 token-shared codebook. (c) Training stage 3: Exclusive latent decoder optimization with frozen switchable token-specific codebooks. Sample face images are from the FFHQ dataset karras2019style.
• Figure 4: Comparisons of different baselines and our methods.
• Figure 5: Visualization analysis from image-level and token-level.