Single-shot lossy compression: mutual information bounds

TL;DR

The paper analyzes one-shot lossy compression under three fidelity constraints by replacing the combinatorial partitioning problem with a convex proxy that minimizes the mutual information $I(X;Y)$ under the corresponding constraint. It establishes tight upper and lower bounds linking the entropy-based objective to the mutual-information proxy, including a sharpened constant $\log_2 e$ and Csiszár-style characterizations of optimal encoders. Results cover guaranteed distortion, conditional excess distortion, and excess distortion, with extensions to randomized encoders and variable error thresholds $\epsilon(X)$. The findings provide a principled, practical surrogate for minimum description length in one-shot settings and clarify the structure of optimal encoding kernels across fidelity regimes.

Abstract

For several styles of fidelity constraints -- guaranteed distortion, conditional excess distortion, excess distortion -- we show mutual information upper bounds on the minimum expected description length needed to represent a random variable. Coupled with the corresponding converses, these results attest that as long as the information content in the data is not too low, minimizing the mutual information under an appropriate fidelity constraint serves as a reasonable proxy for the minimum description length of the data. We provide alternative characterizations of all three convex proxies, shedding light on the structure of their solutions.

Theorems & Definitions (12)

• Theorem 1: guaranteed distortion: quantizer entropy
• proof
• Theorem 2: guaranteed distortion: minimal mutual information
• proof
• Theorem 3: conditional excess distortion: quantizer entropy
• proof
• Theorem 4: conditional excess distortion: minimal mutual information
• proof
• Theorem 5: excess distortion: quantizer entropy
• proof