Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

On the Fluctuations of the Single-Letter $d$-Tilted Sum for Binary Markov Sources

Bhaskar Krishnamachari

TL;DR

It is shown that the centered block sum $J_n(D) - n\mu_D$ is exactly an affine image of the occupation count $N_n = \sum_{t=1}^n \mathbf{1}\{X_t = 1}$ of the Markov chain.

Abstract

The $d$-tilted information has been found to be a useful quantity in finite-blocklength rate-distortion theory for memoryless sources. We study the source-side $d$-tilted sum induced by the single-letter Blahut--Arimoto operating point for a stationary binary Markov source under Hamming distortion; this is a source-side quantity distinct from the $n$-letter operational $d$-tilted information. We show that the centered block sum $J_n(D) - nμ_D$ is exactly an affine image of the occupation count $N_n = \sum_{t=1}^n \mathbf{1}\{X_t = 1\}$ of the Markov chain. As consequences, all centered cumulants are independent of the distortion level~$D$, the finite-$n$ variance admits a closed form, and the exact finite-$n$ distribution and limiting cumulant generating function are given by a $2 \times 2$ transfer matrix.

On the Fluctuations of the Single-Letter $d$-Tilted Sum for Binary Markov Sources

TL;DR

It is shown that the centered block sum is exactly an affine image of the occupation count of the Markov chain.

Abstract

The -tilted information has been found to be a useful quantity in finite-blocklength rate-distortion theory for memoryless sources. We study the source-side -tilted sum induced by the single-letter Blahut--Arimoto operating point for a stationary binary Markov source under Hamming distortion; this is a source-side quantity distinct from the -letter operational -tilted information. We show that the centered block sum is exactly an affine image of the occupation count of the Markov chain. As consequences, all centered cumulants are independent of the distortion level~, the finite- variance admits a closed form, and the exact finite- distribution and limiting cumulant generating function are given by a transfer matrix.
Paper Structure (21 sections, 4 theorems, 34 equations, 1 figure)

This paper contains 21 sections, 4 theorems, 34 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Model and the Binary Hamming Identity
  3. Binary Markov Source
  4. Single-Letter $d$-Tilted Information
  5. The Binary Hamming Identity
  6. Main Theorem
  7. Part (i).
  8. Part (ii).
  9. Part (iii).
  10. Part (iv).
  11. Part (v).
  12. Corollaries
  13. CLT and Berry--Esseen Bound
  14. Symmetric Chains
  15. Finite-$n$ Correction to the Variance
  16. ...and 6 more sections

Key Result

Proposition 2

At the single-letter BA operating point for the binary Markov source under Hamming distortion, in the interior regime $0 < D < \min(\pi_0, \pi_1)$: where $h_2(p) = -p\log_2 p - (1-p)\log_2(1-p)$ is the binary entropy function. In particular, and

Figures (1)

  • Figure 1: Per-letter variance $\mathrm{Var}(J_n)/n$ as a function of blocklength $n$ for the chain $a = 0.1$, $b = 0.3$. The curve converges to $V_{\mathrm{sl}} \approx 1.884$ (dashed) at rate $O(1/n)$ from the i.i.d. baseline $\mathrm{Var}_\pi(\jmath) = 0.471$ (dotted).

Theorems & Definitions (11)

  • Remark 1
  • Proposition 2: Binary Hamming single-letter $d$-tilted information
  • proof
  • Theorem 3: Exact finite-$n$ structure of the binary Hamming $d$-tilted sum
  • proof
  • Remark 4: Where the Markov structure enters
  • Corollary 5: CLT for the $d$-tilted sum
  • proof
  • Corollary 6: Symmetry
  • proof
  • ...and 1 more