A Framework for Robust Lossy Compression of Heavy-Tailed Sources

TL;DR

This work derives the rate-distortion function for $\alpha$-stable sources subject to a constraint on the strength of the error and shows it to be logarithmic in the strength-to-distortion ratio and shows how this framework paves the way for finding optimal quantizers for $\alpha$-stable sources and other general heavy-tailed ones.

Abstract

We study the rate-distortion problem for both scalar and vector memoryless heavy-tailed $α$-stable sources ($0 < α< 2$). Using a recently defined notion of ``strength" as a power measure, we derive the rate-distortion function for $α$-stable sources subject to a constraint on the strength of the error and show it to be logarithmic in the strength-to-distortion ratio. We show how our framework paves the way for finding optimal quantizers for $α$-stable sources and other general heavy-tailed ones. In addition, we study high-rate scalar quantizers and show that uniform ones are asymptotically optimal under the error-strength distortion measure. We compare uniform Gaussian and Cauchy quantizers and show that more representation points for the Cauchy source are required to guarantee the same quantization quality. Our findings generalize the well-known results of rate-distortion and quantization of Gaussian sources ($α= 2$) under a quadratic distortion measure.

Figures (9)

• Figure 1: A plot of $f_{X}(x)$ where $X \sim S(\alpha,1)$ for different values of $\alpha$.
• Figure 2: The rate-distortion function $R(D)$ whenever $X \sim S(\alpha,2)$ for different values of $\alpha$.
• Figure 3: Optimal $M = 2, 3,$ and $4$-points quantizers whenever $X \sim S(1,\gamma_X)$ for different values of $\gamma_X$. (a) Locations of the positive representation points; negative points are symmetric. (b) Resulting minimum strength.
• Figure 4: Optimal $3$-points quantizers whenever $X \sim \mathcal{T}_{\nu}(\gamma)$ for $\nu = 0.7,1,1.3,1.9$. (a) Locations of the positive representation points; negative points are symmetric. (b) Resulting minimum strength.
• Figure 5: The mutual information $I(X,X+N)$ whenever $N \sim S(1,\gamma_N)$ for different values of $\gamma_N$.

Theorems & Definitions (19)

• Definition 1: Univariate Stable Distributions samorodnitsky1996stable
• Definition 2: Equivalent Definition samorodnitsky1996stable
• Remark 1
• Definition 3: Sub-Gaussian Symmetric Alpha-Stable Vector samorodnitsky1996stable
• Definition 4
• Remark 2
• Theorem 1: Rate-Distortion Theorem
• proof
• Theorem 2: Extension to symmetric stable sources
• Theorem 3: Extension to vectors 1