Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Cramer-Rao Bound for Arbitrarily Constrained Sets

Heedong Do, Angel Lozano

TL;DR

This work delivers a universal Cramér-Rao bound for estimating parameters constrained to any set, grounded in the tangent cone ${\mathcal{T}}_{\Theta}(\bm\theta)$. By projecting Fisher information onto the tangent-cone span and incorporating estimator bias, the bound unifies and extends classical constrained CRBs across manifolds, inequalities, and sparsity constraints, while remaining valid for singular or non-smooth cases. The key result yields a bound that depends only on the tangent-cone span via a projection matrix $\bm\Pi$, with explicit forms for nonsingular and arbitrary Fisher information matrices. The framework accommodates transformations of parameters, various constraint types, and composite sets, enabling tight MSE characterizations for a broad class of estimation problems in signal processing and related fields. It also discusses the existence and limitations of unbiased estimators under constraints, highlighting corner-case caveats where the unbiased CRB may fail to be informative.

Abstract

This paper presents a Cramer-Rao bound (CRB) for the estimation of parameters confined to an arbitrary set. Unlike existing results that rely on equality or inequality constraints, manifold structures, or the nonsingularity of the Fisher information matrix, the derived CRB applies to any constrained set and holds for any estimation bias and any Fisher information matrix. The key geometric object governing the new CRB is the tangent cone to the constraint set, whose span determines how the constraints affect the estimation accuracy. This CRB subsumes, unifies, and generalizes known special cases, offering an intuitive and broadly applicable framework to characterize the minimum mean-square error of constrained estimators.

Cramer-Rao Bound for Arbitrarily Constrained Sets

TL;DR

This work delivers a universal Cramér-Rao bound for estimating parameters constrained to any set, grounded in the tangent cone . By projecting Fisher information onto the tangent-cone span and incorporating estimator bias, the bound unifies and extends classical constrained CRBs across manifolds, inequalities, and sparsity constraints, while remaining valid for singular or non-smooth cases. The key result yields a bound that depends only on the tangent-cone span via a projection matrix , with explicit forms for nonsingular and arbitrary Fisher information matrices. The framework accommodates transformations of parameters, various constraint types, and composite sets, enabling tight MSE characterizations for a broad class of estimation problems in signal processing and related fields. It also discusses the existence and limitations of unbiased estimators under constraints, highlighting corner-case caveats where the unbiased CRB may fail to be informative.

Abstract

This paper presents a Cramer-Rao bound (CRB) for the estimation of parameters confined to an arbitrary set. Unlike existing results that rely on equality or inequality constraints, manifold structures, or the nonsingularity of the Fisher information matrix, the derived CRB applies to any constrained set and holds for any estimation bias and any Fisher information matrix. The key geometric object governing the new CRB is the tangent cone to the constraint set, whose span determines how the constraints affect the estimation accuracy. This CRB subsumes, unifies, and generalizes known special cases, offering an intuitive and broadly applicable framework to characterize the minimum mean-square error of constrained estimators.
Paper Structure (17 sections, 7 theorems, 124 equations, 3 figures, 2 tables)

This paper contains 17 sections, 7 theorems, 124 equations, 3 figures, 2 tables.

Table of Contents

  1. Introduction
  2. The Tangent Cone
  3. A Constrained Bound on the Covariance Matrix of the Estimator
  4. Reduction Relative to the Unconstrained CRB
  5. Remarks on the Column Space Condition
  6. Remarks on Transformation of Parameters
  7. The Constrained CRB
  8. Nonsingular Fisher Information Matrix
  9. Arbitrary Fisher Information Matrix
  10. Some Constraint Sets and Their Tangent Cones
  11. Smooth Manifolds
  12. Inequality Constraints
  13. Set of Sparse Vectors
  14. Remarks
  15. Summary and Discussion
  16. ...and 2 more sections

Key Result

Theorem 1

An estimator with bias ${\boldsymbol{b}}$ satisfies and Eq. unconstrained_crb holds with equality if and only if in the mean-square sense.

Figures (3)

  • Figure 1: Set $\Theta$ with the tangent cone $\mathcal{T}_\Theta(\bm \theta)$ shaded in red for different points $\bm \theta$.
  • Figure 2: Tangent cone to $\Theta$ at $\bm \theta$, which for a smooth manifold is identical to the tangent space.
  • Figure 3: The set of sparse vectors for $k=3$ and $s=2$, i.e., $\Theta = \{\bm \theta\in{\mathbb{R}}^3: \|\bm \theta\|_0 \leq 2\}$.

Theorems & Definitions (21)

  • Definition 1
  • Definition 2
  • Theorem 1
  • proof
  • Theorem 2
  • proof
  • Theorem 3
  • proof
  • Lemma 1
  • Example 1
  • ...and 11 more