Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Dispersion of Gaussian Sources with Memory and an Extension to Abstract Sources

Eyyup Tasci, Victoria Kostina

TL;DR

This work extends finite-blocklength rate-distortion analysis to sources with independent but non-identically distributed components, including Gaussian sources with memory under quadratic distortion. It establishes a Gaussian-approximation for the minimal rate: $R(n,d,ε) = \mathbb{R}_n(d) + \sqrt{\mathbb{V}_n(d)/n}\, Q^{-1}(ε) + O(\log n / n)$, with $\mathbb{R}_n(d)$ and $\mathbb{V}_n(d)$ defined from the $n$-letter joint distribution. A key innovation is the point-mass proxy measure that enables typical-set construction and additive analysis across coordinates, extending results beyond iid sources and sharpening Gauss-Markov dispersion bounds. For Gaussian sources with memory, corollaries express the rate and dispersion via finite eigen-spectra and reverse water-filling, yielding explicit formulas and sharper remainder terms. The findings advance finite-blocklength rate-distortion theory to memory-containing Gaussian processes and provide tools for broader non-iid settings.

Abstract

We consider finite blocklength lossy compression of information sources whose components are independent but non-identically distributed. Crucially, Gaussian sources with memory and quadratic distortion can be cast in this form. We show that under the operational constraint of exceeding distortion $d$ with probability at most $ε$, the minimum achievable rate at blocklength $n$ satisfies $R(n, d, ε)=\mathbb{R}_n(d)+\sqrt{\frac{\mathbb{V}_n(d)}{n}}Q^{-1}(ε)+O \left(\frac{\log n}{n}\right)$, where $Q^{-1}(\cdot)$ is the inverse $Q$-function, while $\mathbb{R}_n(d)$ and $\mathbb{V}_n(d)$ are fundamental characteristics of the source computed using its $n$-letter joint distribution and the distortion measure, called the $n$th-order informational rate-distortion function and the source dispersion, respectively. Our result generalizes the existing dispersion result for abstract sources with i.i.d. components. It also sharpens and extends the only known dispersion result for a source with memory, namely, the scalar Gauss-Markov source. The key novel technical tool in our analysis is the point-mass product proxy measure, which enables the construction of typical sets. This proxy generalizes the empirical distribution beyond the i.i.d. setting by preserving additivity across coordinates and facilitating a typicality analysis for sums of independent, non-identical terms.

Dispersion of Gaussian Sources with Memory and an Extension to Abstract Sources

TL;DR

This work extends finite-blocklength rate-distortion analysis to sources with independent but non-identically distributed components, including Gaussian sources with memory under quadratic distortion. It establishes a Gaussian-approximation for the minimal rate: , with and defined from the -letter joint distribution. A key innovation is the point-mass proxy measure that enables typical-set construction and additive analysis across coordinates, extending results beyond iid sources and sharpening Gauss-Markov dispersion bounds. For Gaussian sources with memory, corollaries express the rate and dispersion via finite eigen-spectra and reverse water-filling, yielding explicit formulas and sharper remainder terms. The findings advance finite-blocklength rate-distortion theory to memory-containing Gaussian processes and provide tools for broader non-iid settings.

Abstract

We consider finite blocklength lossy compression of information sources whose components are independent but non-identically distributed. Crucially, Gaussian sources with memory and quadratic distortion can be cast in this form. We show that under the operational constraint of exceeding distortion with probability at most , the minimum achievable rate at blocklength satisfies , where is the inverse -function, while and are fundamental characteristics of the source computed using its -letter joint distribution and the distortion measure, called the th-order informational rate-distortion function and the source dispersion, respectively. Our result generalizes the existing dispersion result for abstract sources with i.i.d. components. It also sharpens and extends the only known dispersion result for a source with memory, namely, the scalar Gauss-Markov source. The key novel technical tool in our analysis is the point-mass product proxy measure, which enables the construction of typical sets. This proxy generalizes the empirical distribution beyond the i.i.d. setting by preserving additivity across coordinates and facilitating a typicality analysis for sums of independent, non-identical terms.
Paper Structure (12 sections, 17 theorems, 119 equations)

This paper contains 12 sections, 17 theorems, 119 equations.

Table of Contents

  1. Introduction
  2. Problem Setup
  3. Operational Definitions
  4. Informational Definitions
  5. Main Results
  6. Converse
  7. Achievability
  8. Conclusion
  9. Berry--Esseen Theorem
  10. Proof of Corollary \ref{['corr:gauss_markov']}
  11. Proof of Lemma \ref{['lem:shell_probability_lower_bound']}
  12. Proof of Lemma \ref{['lem:tilted_concentration']}

Key Result

Theorem 1

Consider a source $\{X_i\}_{i=1}^\infty$ with independent but not identically distributed components under a separable distortion measure. Suppose that assumptions ass:bdd_disp--ass:uni_curv hold. Then for every $\epsilon\in(0,1)$, the minimum achievable rate satisfies

Theorems & Definitions (32)

  • Definition 1
  • Theorem 1: Gaussian approximation
  • Corollary 1
  • proof
  • Corollary 2
  • proof
  • Corollary 3
  • proof : Proof
  • Lemma 1: Kostina and Verdú kostina12_FixedLength, Thm. 7
  • Lemma 2
  • ...and 22 more