Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Nearest-neighbor Entropy Estimators with Weak Metrics

Evgeniy Timofeev, Alexei Kaltchenko

TL;DR

A new nearest-neighbor entropy estimators is constructed and has a parameter with which the estimator is optimized to reduce its bias, and it is shown that estimator's variance is upper-bounded by a nearly optimal Cramer-Rao lower bound.

Abstract

A problem of improving the accuracy of nonparametric entropy estimation for a stationary ergodic process is considered. New weak metrics are introduced and relations between metrics, measures, and entropy are discussed. Based on weak metrics, a new nearest-neighbor entropy estimator is constructed and has a parameter with which the estimator is optimized to reduce its bias. It is shown that estimator's variance is upper-bounded by a nearly optimal Cramer-Rao lower bound.

Nearest-neighbor Entropy Estimators with Weak Metrics

TL;DR

A new nearest-neighbor entropy estimators is constructed and has a parameter with which the estimator is optimized to reduce its bias, and it is shown that estimator's variance is upper-bounded by a nearly optimal Cramer-Rao lower bound.

Abstract

A problem of improving the accuracy of nonparametric entropy estimation for a stationary ergodic process is considered. New weak metrics are introduced and relations between metrics, measures, and entropy are discussed. Based on weak metrics, a new nearest-neighbor entropy estimator is constructed and has a parameter with which the estimator is optimized to reduce its bias. It is shown that estimator's variance is upper-bounded by a nearly optimal Cramer-Rao lower bound.

Paper Structure

This paper contains 5 sections, 9 theorems, 52 equations.

Table of Contents

  1. Notation and Definitions
  2. Metrics on Sequence Spaces
  3. Nearest-neighbor statistics
  4. Entropy Estimator
  5. Conclusion

Key Result

Proposition 1

Let $\mu$ be a shift-invariant ergodic measure on $\Omega$ and $\rho$ be metric rho; then, for $\mu$-almost all points $\boldsymbol{x}\in \Omega$ where $h$ is the entropy of $\mu$.

Theorems & Definitions (14)

  • Proposition 1
  • proof
  • Proposition 2
  • Lemma 3.1
  • proof
  • Corollary 1
  • Theorem 3.2
  • proof
  • Corollary 2
  • proof
  • ...and 4 more