Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Shannon Bounds for Quadratic Rate-Distortion Problems

Michael Gastpar, Erixhen Sula

TL;DR

This paper surveys Shannon bounds on rate-distortion problems under mean-squared error distortion with a particular emphasis on Berger's techniques, encompassing indirect and remote source coding such as the CEO problem, originally proposed by Berger, as well as the Gray-Wyner network as a new contribution.

Abstract

The Shannon lower bound has been the subject of several important contributions by Berger. This paper surveys Shannon bounds on rate-distortion problems under mean-squared error distortion with a particular emphasis on Berger's techniques. Moreover, as a new result, the Gray-Wyner network is added to the canon of settings for which such bounds are known. In the Shannon bounding technique, elegant lower bounds are expressed in terms of the source entropy power. Moreover, there is often a complementary upper bound that involves the source variance in such a way that the bounds coincide in the special case of Gaussian statistics. Such pairs of bounds are sometimes referred to as Shannon bounds. The present paper puts Berger's work on many aspects of this problem in the context of more recent developments, encompassing indirect and remote source coding such as the CEO problem, originally proposed by Berger, as well as the Gray-Wyner network as a new contribution.

Shannon Bounds for Quadratic Rate-Distortion Problems

TL;DR

This paper surveys Shannon bounds on rate-distortion problems under mean-squared error distortion with a particular emphasis on Berger's techniques, encompassing indirect and remote source coding such as the CEO problem, originally proposed by Berger, as well as the Gray-Wyner network as a new contribution.

Abstract

The Shannon lower bound has been the subject of several important contributions by Berger. This paper surveys Shannon bounds on rate-distortion problems under mean-squared error distortion with a particular emphasis on Berger's techniques. Moreover, as a new result, the Gray-Wyner network is added to the canon of settings for which such bounds are known. In the Shannon bounding technique, elegant lower bounds are expressed in terms of the source entropy power. Moreover, there is often a complementary upper bound that involves the source variance in such a way that the bounds coincide in the special case of Gaussian statistics. Such pairs of bounds are sometimes referred to as Shannon bounds. The present paper puts Berger's work on many aspects of this problem in the context of more recent developments, encompassing indirect and remote source coding such as the CEO problem, originally proposed by Berger, as well as the Gray-Wyner network as a new contribution.
Paper Structure (19 sections, 12 theorems, 74 equations, 5 figures)

This paper contains 19 sections, 12 theorems, 74 equations, 5 figures.

Table of Contents

  1. Introduction
  2. Contributions
  3. Related Work
  4. Notation
  5. MMSE Estimation
  6. The Quadratic Rate-Distortion Function
  7. The Scalar Case
  8. The Vector Case
  9. The Conditional Rate-Distortion Function and the Wyner-Ziv Problem
  10. Remote (Indirect) Source Coding
  11. The Gray-Wyner Source Coding Network
  12. The AWGN CEO Problem
  13. Discussion and Open Problems
  14. Berger's variational theorem
  15. Proofs of \ref{['lem:cond-rate-distor']} and \ref{['lem:cond-rate-distor-WZ']}
  16. ...and 4 more sections

Key Result

Theorem 1

For a continuous random variable $X$ and mean-squared error distortion $d(x,\hat{x})=(x-\hat{x})^2,$ the rate-distortion function is bounded by

Figures (5)

  • Figure 1: The conditional rate-distortion problem.
  • Figure 2: The Wyner-Ziv rate-distortion problem.
  • Figure 3: The Remote Rate-Distortion Problem
  • Figure 4: The Gray-Wyner Network
  • Figure 5: The $M$-agent CEO problem. $X$ is an arbitrary source with variance (power) $\sigma_X^2$ (not necessarily Gaussian) and entropy power $\operatorname{N} \lparen*\rparen{X}.$ For the version considered in the present paper, the source observation kernels $p(y_m|x)$ consist in adding independent Gaussian noises $Z_m$ of variance $\sigma_Z^2.$

Theorems & Definitions (12)

  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Theorem 4
  • Theorem 5
  • Theorem 6
  • Corollary 1
  • Theorem 7
  • Theorem 8: Corollary 2 in EswaranG:19
  • Theorem 9: Theorem 4.2.3 in BergerBook
  • ...and 2 more