Ziv-Merhav estimation for hidden-Markov processes

TL;DR

A proof of strong consistency of a Ziv-Merhav-type estimator of the cross entropy rate for pairs of hidden-Markov processes using tools from the thermodynamic formalism is presented.

Abstract

We present a proof of strong consistency of a Ziv-Merhav-type estimator of the cross entropy rate for pairs of hidden-Markov processes. Our proof strategy has two novel aspects: the focus on decoupling properties of the laws and the use of tools from the thermodynamic formalism.

Figures (2)

• Figure 1: The pressure may or may not be differentiable, but its left derivative at the origin can still be identified as the cross entropy rate under quite general assumptions. As a visual aid to the convexity argument for Lemma \ref{['lem:ing-IV']}, the gray dashed line (figuratively) has slope $D_-\bar{q}(0)-\tfrac{\epsilon}{4}$.
• Figure 2: For single realizations of HMPs $\bf{X}$ and $\bf{Y}$, we plot different estimators of $h^{\textnormal{c}}(\mathbb{P}_{\mathbf{Y}}|\mathbb{P}_{\mathbf{X}})$ up to $N = 2^{20}$ (Top). Over 32 realizations, we plot our estimation of the RMSE of the different estimators up to $N = 2^{17}$ (Bottom).

Theorems & Definitions (15)

• Definition 1
• Theorem 2
• Lemma 3
• Lemma 4
• Definition 5
• Lemma 6
• proof : Proof sketch
• Lemma 7
• proof : Proof sketch
• Lemma 8