Optimal Neural Compressors for the Rate-Distortion-Perception Tradeoff

TL;DR

This work addresses the neural rate-distortion-perception (RDP) tradeoff by proposing lattice transform coding (LTC) based compressors that incorporate dithering under varying shared randomness budgets. The authors introduce SD-LTC (infinite shared randomness), PD-LTC (no shared randomness), and QSD-LTC (finite shared randomness via nested lattices), and prove that SD-LTC achieves the RDP function $R(D,P)$ for Gaussian sources while PD-LTC achieves $R(D/2,\infty)$ at $P=0$; QSD-LTC provides a practical continuum between these regimes. Empirical results on synthetic Gaussian data and real-world datasets (e.g., MNIST, Physics, Speech) show that increasing shared randomness and lattice packing efficiency improves the RDP performance, with QSD-LTC closely approaching SD-LTC for modest randomness budgets. The findings offer a scalable, low-complexity path to RDP-optimal neural compressors and highlight the tradeoffs between randomness, lattice quality, and computational efficiency for perceptually aware compression.

Abstract

Recent efforts in neural compression have focused on the rate-distortion-perception (RDP) tradeoff, where the perception constraint ensures the source and reconstruction distributions are close in terms of a statistical divergence. Theoretical work on RDP describes properties of RDP-optimal compressors without providing constructive and low complexity solutions. While classical rate distortion theory shows that optimal compressors should efficiently pack space, RDP theory additionally shows that infinite randomness shared between the encoder and decoder may be necessary for RDP optimality. In this paper, we propose neural compressors that are low complexity and benefit from high packing efficiency through lattice coding and shared randomness through shared dithering over the lattice cells. For two important settings, namely infinite shared and zero shared randomness, we analyze the RDP tradeoff achieved by our proposed neural compressors and show optimality in both cases. Experimentally, we investigate the roles that these two components of our design, lattice coding and randomness, play in the performance of neural compressors on synthetic and real-world data. We observe that performance improves with more shared randomness and better lattice packing.

Figures (12)

• Figure 1: Lattice transform coding (LTC) with $R_c$ bits of (shared) randomness using dithering; $\bm{u} \sim \mathrm{Unif}(\mathcal{V}_0(\Lambda))$ and $\bm{u}_f \sim \mathrm{Unif}(\mathcal{V}_0(\Lambda_f))$ are continuous, and $\hat{\bm{u}} \sim \mathrm{Unif}(|\Lambda / \Lambda_f|)$ is discrete, where $\Lambda, \Lambda_f$ are nested lattices. LTC/PD-LTC entropy-code $Q_{\Lambda}(\bm{y})$ with likelihoods $p_{\hat{\bm{y}}}$. SD-LTC and QSD-LTC entropy-code $Q_{\Lambda}(\bm{y} - \bm{u})$ and $Q_{\Lambda}(\bm{y} - \hat{\bm{u}})$ with likelihoods $p_{\bm{c}|\bm{u}}$ and $p_{\bm{c}|\hat{\bm{u}}}$, respectively.
• Figure 2: Reconstr. of $\bm{x}$ under PD (gray) or SD (blue); $s=1$.
• Figure 3: In the latent space, SD-LTC (left) becomes AWGN-like. PD-LTC (right) models the sum of a lattice Gaussian and Gaussian; $s$ must be large enough for the sum to be Gaussian.
• Figure 4: Effect of lattice choice and shared randomness on Gaussians at $P=0$.
• Figure 5: LTC, PD-LTC, and SD-LTC on real-world sources; R-D (top) and R-P (bottom).

Theorems & Definitions (35)

• Definition 3.1: Shared-Dither Lattice Transform Code (SD-LTC); Fig. \ref{['fig:SD-LTC']}
• Definition 3.2: Private-Dither Lattice Transform Code (PD-LTC); Fig. \ref{['fig:PD-LTC']}
• Proposition 3.3
• Definition 3.4: Quantized Shared-Dither Lattice Transform Code (QSD-LTC); Fig. \ref{['fig:QSD-LTC']}
• Remark 3.5
• Remark 3.6
• Proposition 4.1: RDP function for Gaussian source zhang2021universal
• Remark 4.2
• Theorem 4.3: Optimality of SD-LTC for Gaussian source
• Remark 4.4