Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Redundancy of Non-Singular Channel Simulation

Gergely Flamich, Sharang M. Sriramu, Aaron B. Wagner

TL;DR

The paper investigates the fundamental limits of channel simulation for lossy source coding, establishing a tight one-shot lower bound on the average communication cost in terms of the channel simulation divergence $D_{CS}$. It proves a matching second-order asymptotic bound for iid, non-singular channels, showing that the redundancy scales as $\frac{\mathrm{lb}(n)}{2n}$. Two complementary proofs are provided: a direct one-shot argument based on $D_{CS}$ and an independent large-deviations approach using tilted measures and cumulants. Together, these results resolve the open non-discrete/non-singular case and clarify the role of channel structure in the redundancy of channel simulation, with implications for rate-distortion-perception tradeoffs in learning-inspired compression schemes.

Abstract

Channel simulation is an alternative to quantization and entropy coding for performing lossy source coding. Recently, channel simulation has gained significant traction in both the machine learning and information theory communities, as it integrates better with machine learning-based data compression algorithms and has better rate-distortion-perception properties than quantization. As the practical importance of channel simulation increases, it is vital to understand its fundamental limitations. Recently, Sriramu and Wagner provided an almost complete characterisation of the redundancy of channel simulation algorithms. In this paper, we complete this characterisation. First, we significantly extend a result of Li and El Gamal, and show that the redundancy of any instance of a channel simulation problem is lower bounded by the channel simulation divergence. Second, we give two proofs that the asymptotic redundancy of simulating iid non-singular channels is lower-bounded by $1/2$: one using a direct approach based on the asymptotic expansion of the channel simulation divergence and one using large deviations theory.

The Redundancy of Non-Singular Channel Simulation

TL;DR

The paper investigates the fundamental limits of channel simulation for lossy source coding, establishing a tight one-shot lower bound on the average communication cost in terms of the channel simulation divergence . It proves a matching second-order asymptotic bound for iid, non-singular channels, showing that the redundancy scales as . Two complementary proofs are provided: a direct one-shot argument based on and an independent large-deviations approach using tilted measures and cumulants. Together, these results resolve the open non-discrete/non-singular case and clarify the role of channel structure in the redundancy of channel simulation, with implications for rate-distortion-perception tradeoffs in learning-inspired compression schemes.

Abstract

Channel simulation is an alternative to quantization and entropy coding for performing lossy source coding. Recently, channel simulation has gained significant traction in both the machine learning and information theory communities, as it integrates better with machine learning-based data compression algorithms and has better rate-distortion-perception properties than quantization. As the practical importance of channel simulation increases, it is vital to understand its fundamental limitations. Recently, Sriramu and Wagner provided an almost complete characterisation of the redundancy of channel simulation algorithms. In this paper, we complete this characterisation. First, we significantly extend a result of Li and El Gamal, and show that the redundancy of any instance of a channel simulation problem is lower bounded by the channel simulation divergence. Second, we give two proofs that the asymptotic redundancy of simulating iid non-singular channels is lower-bounded by : one using a direct approach based on the asymptotic expansion of the channel simulation divergence and one using large deviations theory.
Paper Structure (22 sections, 13 theorems, 87 equations)

This paper contains 22 sections, 13 theorems, 87 equations.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. One-shot lower bound
  4. Proof of the Asymptotic Result Using the Channel Simulation Divergence
  5. Proof of the Asymptotic Result Using Large Deviations Techniques
  6. Cumulants and exponentially tilted measures
  7. Additional Assumptions
  8. Sketch of the Main Argument
  9. Initial Rate Bound
  10. Restriction to Typical $(Y^n,Z)$ Realizations
  11. Reduction to Ball-Like Sets
  12. Change of Measure and Conclusion
  13. Discussion
  14. Author Contributions
  15. Additional Computations
  16. ...and 7 more sections

Key Result

Theorem 3.1

Let $X, Y \sim P_{X, Y}$ be a pair of dependent, Polish random variables. Then, for any common randomness $Z$ that admits a functional representation of $Y$, i.e., such that $Z \perp X$ and $Y = g(X, Z)$ for some measurable function $g$:

Theorems & Definitions (35)

  • Definition 3.1: Width function, Channel simulation divergence goc2024channel
  • Theorem 3.1: A one-shot lower bound on the efficiency of channel simulation
  • proof
  • Lemma 4.1
  • proof
  • Theorem 4.2: Asymptotic Redundancy of Non-singular Channels
  • proof
  • Lemma 4.3
  • proof
  • Theorem 5.1
  • ...and 25 more