Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Improving Cramér-Rao Bound And Its Variants: An Extrinsic Geometry Perspective

Sunder Ram Krishnan

Abstract

This work presents a geometric refinement of the classical Cramér--Rao bound (CRB) in the non-asymptotic regime by incorporating curvature-aware corrections based on the second fundamental form associated with the statistical model manifold. That is, our formulation shows that relying on the extrinsic geometry of the square root embedding of the manifold in the ambient Hilbert space comprising square integrable functions with respect to a fixed base measure offers a rigorous (and intuitive) way to improve upon the CRB and some of its variants, such as the Bhattacharyya-type bounds, that use higher-order derivatives of the log-likelihood. Precisely, the improved bounds in the latter case make explicit use of the elegant framework offered by employing the Faà di Bruno formula and exponential Bell polynomials in expressing the jets associated with the square root embedding in terms of the raw scores. The interplay between the geometry of the statistical embedding and the behavior of the estimator variance is quantitatively analyzed in concrete examples, showing that our corrections can meaningfully tighten the lower bound, suggesting further exploration into connections with estimator efficiency in more general situations.

Improving Cramér-Rao Bound And Its Variants: An Extrinsic Geometry Perspective

Abstract

This work presents a geometric refinement of the classical Cramér--Rao bound (CRB) in the non-asymptotic regime by incorporating curvature-aware corrections based on the second fundamental form associated with the statistical model manifold. That is, our formulation shows that relying on the extrinsic geometry of the square root embedding of the manifold in the ambient Hilbert space comprising square integrable functions with respect to a fixed base measure offers a rigorous (and intuitive) way to improve upon the CRB and some of its variants, such as the Bhattacharyya-type bounds, that use higher-order derivatives of the log-likelihood. Precisely, the improved bounds in the latter case make explicit use of the elegant framework offered by employing the Faà di Bruno formula and exponential Bell polynomials in expressing the jets associated with the square root embedding in terms of the raw scores. The interplay between the geometry of the statistical embedding and the behavior of the estimator variance is quantitatively analyzed in concrete examples, showing that our corrections can meaningfully tighten the lower bound, suggesting further exploration into connections with estimator efficiency in more general situations.

Paper Structure

This paper contains 30 sections, 7 theorems, 162 equations, 1 figure.

Table of Contents

  1. Introduction
  2. Preliminaries
  3. Notation, square root embedding, and assumptions
  4. Geometric preliminaries
  5. Ambient (flat) derivative and induced connection
  6. Second fundamental form (vectors) and normal bundle
  7. Faà di Bruno and exponential Bell polynomials
  8. Exponential (complete) Bell polynomials.
  9. Faà di Bruno (exponential form).
  10. Application to the square root embedding.
  11. Computation of the Gram matrix $G$ and vector $v$.
  12. Lower-order expansions.
  13. Outlook and road ahead
  14. Curvature-corrected variance bounds -- CRB case
  15. Higher-order CRB and curvature corrections in the square root embedding
  16. ...and 15 more sections

Key Result

Theorem 2

Assume the model satisfies Assumption ass:regularity. Fix the true parameter value $\theta$ and let $T(X)$ be an estimator of $\theta$ which is unbiased (in a neighbourhood of $\theta$). Define the centered error With $s_\theta=\sqrt{f(\cdot;\theta)}\in L^2(\mu)$, consider the ambient error vector, the one-dimensional tangent space, and residual, respectively: Finally denote the Fisher informati

Figures (1)

  • Figure 1: The ratio $\frac{\langle H_3 s_0, W_2^\perp \rangle^2}{\|W_2^\perp\|^2}$ as a function of the parameter $c$.

Theorems & Definitions (23)

  • Remark 1
  • Theorem 2: Residual variance and CRB refinement, $m=1$
  • proof
  • Theorem 3: Hilbert space characterization of higher-order CRB b1946
  • proof
  • Remark 2
  • Lemma 4: square root isometry and score-image representation
  • proof
  • Example 1: Saturation at $m = 3$ in a symmetric quartic location model
  • Theorem 5: Curvature-corrected variance bound, $m\ge1$
  • ...and 13 more