Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Lossless Analog Compression

Giovanni Alberti, Helmut Bölcskei, Camillo De Lellis, Günther Koliander, Erwin Riegler

TL;DR

The paper establishes fundamental limits for lossless analog compression in a nonasymptotic setting where one seeks zero-error recovery of an $m$-dimensional real vector from $n$ linear measurements $y=Ax$. It introduces the lower modified Minkowski dimension $K(x)$ to quantify description complexity and proves that $n>K(x)$ suffices for zero-error recovery via a Borel measurable decoder, with $s$-rectifiable vectors yielding $K(x)\le s$. It further characterizes a strong converse through the notion of $s$-analytic random vectors, showing that for these, $n\ge s$ is necessary to achieve probability of error less than 1, while highlighting that $n<s$ can sometimes suffice for specific geometric constructions (e.g., certain $2$-rectifiable sets). The work leverages geometric measure theory and real analytic function theory to bridge dimension-based descriptions, rectifiability, and analytic structure, providing both achievability and counterexamples that delineate the precise limits of lossless analog compression.

Abstract

We establish the fundamental limits of lossless analog compression by considering the recovery of arbitrary m-dimensional real random vectors x from the noiseless linear measurements y=Ax with n x m measurement matrix A. Our theory is inspired by the groundbreaking work of Wu and Verdu (2010) on almost lossless analog compression, but applies to the nonasymptotic, i.e., fixed-m case, and considers zero error probability. Specifically, our achievability result states that, for almost all A, the random vector x can be recovered with zero error probability provided that n > K(x), where K(x) is given by the infimum of the lower modified Minkowski dimension over all support sets U of x. We then particularize this achievability result to the class of s-rectifiable random vectors as introduced in Koliander et al. (2016); these are random vectors of absolutely continuous distribution -- with respect to the s-dimensional Hausdorff measure -- supported on countable unions of s-dimensional differentiable submanifolds of the m-dimensional real coordinate space. Countable unions of differentiable submanifolds include essentially all signal models used in the compressed sensing literature. Specifically, we prove that, for almost all A, s-rectifiable random vectors x can be recovered with zero error probability from n>s linear measurements. This threshold is, however, found not to be tight as exemplified by the construction of an s-rectifiable random vector that can be recovered with zero error probability from n<s linear measurements. This leads us to the introduction of the new class of s-analytic random vectors, which admit a strong converse in the sense of n greater than or equal to s being necessary for recovery with probability of error smaller than one. The central conceptual tools in the development of our theory are geometric measure theory and the theory of real analytic functions.

Lossless Analog Compression

TL;DR

The paper establishes fundamental limits for lossless analog compression in a nonasymptotic setting where one seeks zero-error recovery of an -dimensional real vector from linear measurements . It introduces the lower modified Minkowski dimension to quantify description complexity and proves that suffices for zero-error recovery via a Borel measurable decoder, with -rectifiable vectors yielding . It further characterizes a strong converse through the notion of -analytic random vectors, showing that for these, is necessary to achieve probability of error less than 1, while highlighting that can sometimes suffice for specific geometric constructions (e.g., certain -rectifiable sets). The work leverages geometric measure theory and real analytic function theory to bridge dimension-based descriptions, rectifiability, and analytic structure, providing both achievability and counterexamples that delineate the precise limits of lossless analog compression.

Abstract

We establish the fundamental limits of lossless analog compression by considering the recovery of arbitrary m-dimensional real random vectors x from the noiseless linear measurements y=Ax with n x m measurement matrix A. Our theory is inspired by the groundbreaking work of Wu and Verdu (2010) on almost lossless analog compression, but applies to the nonasymptotic, i.e., fixed-m case, and considers zero error probability. Specifically, our achievability result states that, for almost all A, the random vector x can be recovered with zero error probability provided that n > K(x), where K(x) is given by the infimum of the lower modified Minkowski dimension over all support sets U of x. We then particularize this achievability result to the class of s-rectifiable random vectors as introduced in Koliander et al. (2016); these are random vectors of absolutely continuous distribution -- with respect to the s-dimensional Hausdorff measure -- supported on countable unions of s-dimensional differentiable submanifolds of the m-dimensional real coordinate space. Countable unions of differentiable submanifolds include essentially all signal models used in the compressed sensing literature. Specifically, we prove that, for almost all A, s-rectifiable random vectors x can be recovered with zero error probability from n>s linear measurements. This threshold is, however, found not to be tight as exemplified by the construction of an s-rectifiable random vector that can be recovered with zero error probability from n<s linear measurements. This leads us to the introduction of the new class of s-analytic random vectors, which admit a strong converse in the sense of n greater than or equal to s being necessary for recovery with probability of error smaller than one. The central conceptual tools in the development of our theory are geometric measure theory and the theory of real analytic functions.

Paper Structure

This paper contains 24 sections, 76 theorems, 259 equations, 3 figures.

Table of Contents

  1. Introduction
  2. Achievability
  3. Rectifiable Sets and Rectifiable Random Vectors
  4. Strong Converse
  5. Proof of Theorem \ref{['th1']} (Achievability)
  6. Existence of Borel Measurable $g$
  7. Proof of Theorem \ref{['prpcounter']}
  8. Proof of Theorem \ref{['thm.converse']} (Strong Converse)
  9. Proof of \ref{['posH1']}--\ref{['eq:Jhz0']}
  10. Proof of Lemma \ref{['LemmaExarec']}
  11. Proof of Lemma \ref{['lem:recsetlip']}
  12. Proof of Lemma \ref{['lem:uniques']}
  13. Proof of Lemma \ref{['lem:convimpliesA']}
  14. Proof of Lemma \ref{['lem:propanalytic']}
  15. Proof of Lemma \ref{['lemancomp']}
  16. ...and 9 more sections

Key Result

Lemma 2.1

Let ${\boldsymbol{\mathsf{x}}}\xspace\in\mathbb R\xspace^k$ and $f\colon \mathbb R\xspace^k\to \mathbb R\xspace^m$ be locally Lipschitz. Then, $K(f({\boldsymbol{\mathsf{x}}}\xspace))\leq K({\boldsymbol{\mathsf{x}}}\xspace)$.

Figures (3)

  • Figure 1: For every vector ${\boldsymbol{h}}\xspace\subseteq\mathbb R\xspace^2\!\setminus\!\{{\boldsymbol{0}}\xspace\}$, the linear subspace $\mathcal{T}\xspace_{\boldsymbol{h}}\xspace:=\{{\boldsymbol{h}}\xspace z:z\in\mathbb R\xspace\}$ intersects the unit circle $\mathcal{S}\xspace_1$ in the two antipodal points $\pm{\boldsymbol{h}}\xspace/\|{\boldsymbol{h}}\xspace\|_2$.
  • Figure 2: The set $Q_2$ consists of the four grey squares. The set $Q_3$ consists of the sixteen shaded black squares.
  • Figure 3: ( fa14) Graph of $\mathscr{H}^{d}(\mathcal{U}\xspace)$ as a function of $d\in[0,m]$ for a set $\mathcal{U}\xspace\subseteq\mathbb R\xspace^{m}$.

Theorems & Definitions (159)

  • Definition 2.1
  • Definition 2.2
  • Example 2.1
  • Example 2.2
  • Lemma 2.1
  • proof
  • Corollary 2.1
  • proof
  • Theorem 2.1
  • proof
  • ...and 149 more