Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Rate-Distortion Analysis for Composite Sources Under Subsource-Dependent Fidelity Criteria

Jiakun Liu, H. Vincent Poor, Iickho Song, Wenyi Zhang

TL;DR

This work proposes subsource-dependent fidelity criteria for composite sources and uses them to formulate a rate-distortion problem, and obtains a single-letter expression for the rate-distortion function.

Abstract

A composite source, consisting of multiple subsources and a memoryless switch, outputs one symbol at a time from the subsource selected by the switch. If some data should be encoded more accurately than other data from an information source, the composite source model is suitable because in this model different distortion constraints can be put on the subsources. In this context, we propose subsource-dependent fidelity criteria for composite sources and use them to formulate a rate-distortion problem. We solve the problem and obtain a single-letter expression for the rate-distortion function. Further rate-distortion analysis characterizes the performance of classify-then-compress (CTC) coding, which is frequently used in practice when subsource-dependent fidelity criteria are considered. Our analysis shows that CTC coding generally has performance loss relative to optimal coding, even if the classification is perfect. We also identify the cause of the performance loss, that is, class labels have to be reproduced in CTC coding. Last but not least, we show that the performance loss is negligible for asymptotically small distortion if CTC coding is appropriately designed and some mild conditions are satisfied.

A Rate-Distortion Analysis for Composite Sources Under Subsource-Dependent Fidelity Criteria

TL;DR

This work proposes subsource-dependent fidelity criteria for composite sources and uses them to formulate a rate-distortion problem, and obtains a single-letter expression for the rate-distortion function.

Abstract

A composite source, consisting of multiple subsources and a memoryless switch, outputs one symbol at a time from the subsource selected by the switch. If some data should be encoded more accurately than other data from an information source, the composite source model is suitable because in this model different distortion constraints can be put on the subsources. In this context, we propose subsource-dependent fidelity criteria for composite sources and use them to formulate a rate-distortion problem. We solve the problem and obtain a single-letter expression for the rate-distortion function. Further rate-distortion analysis characterizes the performance of classify-then-compress (CTC) coding, which is frequently used in practice when subsource-dependent fidelity criteria are considered. Our analysis shows that CTC coding generally has performance loss relative to optimal coding, even if the classification is perfect. We also identify the cause of the performance loss, that is, class labels have to be reproduced in CTC coding. Last but not least, we show that the performance loss is negligible for asymptotically small distortion if CTC coding is appropriately designed and some mild conditions are satisfied.
Paper Structure (28 sections, 19 theorems, 79 equations, 8 figures)

This paper contains 28 sections, 19 theorems, 79 equations, 8 figures.

Table of Contents

  1. Introduction
  2. Related Works
  3. Notation
  4. Definitions
  5. Subsource-Dependent Fidelity Criteria
  6. Variable-Length Codes and CTC Coding
  7. RD Functions
  8. Main Results
  9. Coding Theorems
  10. Properties of RD Functions
  11. CTC Code Where Labels are Compressed by a Variable-Length Code
  12. A Case with Perfect Classification
  13. Comparison of RD Functions
  14. Proof Sketches of Coding Theorems
  15. Theorem \ref{['thm:results.theorems.general']}
  16. ...and 13 more sections

Key Result

Theorem 1

If $R > R^{*} ( D )$, then there exists an $( \mathcal{X} , \hat{\mathcal{X}} )$-variable-length code $( f , \varphi )$ achieving $( R , D )$ for $( S , X )$ and $\{ ( d_{\lambda} , \mathcal{S}_{\lambda} ) \}_{\lambda \in \Lambda}$, and satisfying Conversely, if an $( \mathcal{X} , \hat{\mathcal{X}} )$-variable-length code $( f , \varphi )$ achieves $( R , D )$ for $( S , X )$ and $\{ ( d_{\lambd

Figures (8)

  • Figure 1: A composite source with $L$ subsources.
  • Figure 2: An example of CTC coding.
  • Figure 3: The transmitted bit string in another implementation of CTC coding with $\mathcal{U} = ( 3 ]$.
  • Figure 4: The relation between $R^{\mathrm{O}}$ and $R^{*}$ in Example \ref{['exp:results.properties.omos']}. Each curve shows the relation between $D ( 0 )$ and $R^{*} ( D )$ under the constraint $\mathrm{Pr} [ S = 0 ] D ( 0 ) + \mathrm{Pr} [ S = 1 ] D ( 1 ) = \delta$.
  • Figure 5: The RD function $R^{*}$ in Example \ref{['exp:results.case.hamming']}.
  • ...and 3 more figures

Theorems & Definitions (26)

  • Definition 1
  • Definition 2
  • Definition 3
  • Definition 4
  • Theorem 1
  • Theorem 2
  • Theorem 3
  • Corollary 1
  • Corollary 2
  • Corollary 3
  • ...and 16 more