Universal Representations for Classification-enhanced Lossy Compression

TL;DR

The paper tackles efficient lossy compression under multiple objectives by proposing universal representations: a single encoder that supports diverse decoding goals across rate–distortion–perception and rate–distortion–classification constraints. It formalizes these tradeoffs with RDPC functions and introduces a universal RDC framework where decoders are trained for different targets while a fixed encoder provides the shared representation, using dithered quantization and GAN/classifier regularization. Empirical results on MNIST show that universal encoders can closely match end-to-end performance for perception-related objectives, but face distortion penalties in the RDC setting when reused across tradeoffs; scaling the decoder tuning parameters helps mitigate this. Overall, the approach offers a practical path to reducing training cost and model redundancy, with potential extensions to higher-resolution image and video compression.

Abstract

In lossy compression, the classical tradeoff between compression rate and reconstruction distortion has traditionally guided algorithm design. However, Blau and Michaeli [5] introduced a generalized framework, known as the rate-distortion-perception (RDP) function, incorporating perceptual quality as an additional dimension of evaluation. More recently, the rate-distortion-classification (RDC) function was investigated in [19], evaluating compression performance by considering classification accuracy alongside distortion. In this paper, we explore universal representations, where a single encoder is developed to achieve multiple decoding objectives across various distortion and classification (or perception) constraints. This universality avoids retraining encoders for each specific operating point within these tradeoffs. Our experimental validation on the MNIST dataset indicates that a universal encoder incurs only minimal performance degradation compared to individually optimized encoders for perceptual image compression tasks, aligning with prior results from [23]. Nonetheless, we also identify that in the RDC setting, reusing an encoder optimized for one specific classification-distortion tradeoff leads to a significant distortion penalty when applied to alternative points.

Figures (7)

• Figure 1: Schematic representation of a task-oriented lossy compression framework.
• Figure 2: Illustration of the universal representation framework.
• Figure 3: Diagram of the experimental framework for the universal representation model. Initially, an encoder network $f$ is trained to achieve a predetermined balance between classification accuracy and reconstruction distortion (alternatively, perception and distortion). After training, the encoder's parameters are fixed. Subsequently, multiple specialized decoders $\{g_i\}$ are independently optimized, each targeting distinct trade-off criteria using the fixed representation $z$ generated by encoder $f$. A shared source of randomness $u$ is accessible to both sender and receiver to facilitate universal quantization. Additionally, dedicated critic networks $\{h_i\}$ are concurrently trained alongside each decoder to enhance perceptual quality. A pre-trained classifier network ($C$) is utilized for evaluating classification performance.
• Figure 4: Classification-distortion-rate functions along various rates for the MNIST dataset, illustrating the tradeoff between rate, distortion, and classification.
• Figure 5: Perception-distortion-rate functions evaluated at a fixed rate of $R = 4.75$ on the MNIST dataset. Points highlighted with black outlines indicate results obtained from end-to-end trained encoder-decoder models tailored specifically to particular perception-distortion targets. All other points represent outcomes from universal models, in which decoders are trained separately using representations from an encoder fixed at low perceptual distortion ($\lambda_p=0.015$). The universal models closely match the performance of the jointly trained models across the entire range of trade-offs.