Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Information Geometry Approach to Parameter Estimation in Hidden Markov Models

Masahito Hayashi

TL;DR

This work focuses on a partial observation model with Markovian process and shows that the asymptotic estimation error of this model is given as the inverse of projective Fisher information of transition matrices, and proposes a novel method to estimate hidden Markovians process.

Abstract

We consider the estimation of the transition matrix of a hidden Markovian process by using information geometry with respect to transition matrices. In this paper, only the histogram of $k$-memory data is used for the estimation. To establish our method, we focus on a partial observation model with the Markovian process and we propose an efficient estimator whose asymptotic estimation error is given as the inverse of projective Fisher information of transition matrices. This estimator is applied to the estimation of the transition matrix of the hidden Markovian process. In this application, we carefully discuss the equivalence problem for hidden Markovian process on the tangent space.

Information Geometry Approach to Parameter Estimation in Hidden Markov Models

TL;DR

This work focuses on a partial observation model with Markovian process and shows that the asymptotic estimation error of this model is given as the inverse of projective Fisher information of transition matrices, and proposes a novel method to estimate hidden Markovians process.

Abstract

We consider the estimation of the transition matrix of a hidden Markovian process by using information geometry with respect to transition matrices. In this paper, only the histogram of -memory data is used for the estimation. To establish our method, we focus on a partial observation model with the Markovian process and we propose an efficient estimator whose asymptotic estimation error is given as the inverse of projective Fisher information of transition matrices. This estimator is applied to the estimation of the transition matrix of the hidden Markovian process. In this application, we carefully discuss the equivalence problem for hidden Markovian process on the tangent space.

Paper Structure

This paper contains 31 sections, 33 theorems, 79 equations, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Summary of results
  3. Estimation in partial observation model
  4. Estimation in hidden Markovian model
  5. Exponential family of transition matrices
  6. Preparation
  7. Linear independence
  8. Exponential family
  9. Mixture family
  10. Divergence
  11. Central limit theorem
  12. Partial observation model
  13. $k-1$-memory transition matrices
  14. $k-1$-memory transition matrix and joint distribution
  15. Exponential family of $k-1$-memory transition matrices
  16. ...and 16 more sections

Key Result

Theorem 2.1

Under regularity conditions C1, C2, and C3 given in Section s4, the distribution of the estimation error of the estimator $\hat{\theta}_{2,3}(\vec{Y}_{1,2}^n)$ converges to the zero-mean Gaussian distribution whose variance is the inverse of the projective Fisher information, whose definition wil

Figures (2)

  • Figure 1: Pair of transition matrices: The transition matrix $W$ determines the Markovian process on the set ${\cal X}$ of hidden states. The transition matrix $V$ determines the observed variable $Y$ with the condition on the hidden variable $X$. If $k$ is sufficiently large, the joint distribution on $Y_1, \ldots, Y_{k+1}$ uniquely determines the transition matrices $W$ and $V$. The partial observation model is applied to the $k+1$ joint system on $X_{1},Y_2,\ldots, X_{k+1},Y_{k+2}$.
  • Figure 2: Estimator of partial observation model

Theorems & Definitions (43)

  • Theorem 2.1
  • Theorem 2.2
  • Theorem 2.3
  • Lemma 3.1
  • proof
  • Proposition 3.2: HW-est
  • Proposition 3.3: HW-est
  • Lemma 3.4
  • proof
  • Lemma 3.5
  • ...and 33 more