Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learned Bayesian Cramér-Rao Bound for Unknown Measurement Models Using Score Neural Networks

Hai Victor Habi, Hagit Messer, Yoram Bresler

TL;DR

This work tackles computing a Bayesian Cramér-Rao bound (BCRB) when priors and measurement models are unknown by introducing two fully learned LBCRB frameworks. The Posterior Approach learns the posterior score and constructs the bound from data, while the Measurement-Prior Approach decomposes the bound into prior and Fisher-score components and integrates domain knowledge via a Physics-encoded Score Neural Network (PeSNN) and Fisher Score Matching. Theoretical results establish consistency and finite-sample error bounds for both methods, and experiments across linear and nonlinear problems (including quantization and underwater noise) demonstrate that the Measurement-Prior Approach often achieves lower sample complexity and higher accuracy, especially when physical models are available. The work enables practical, data-driven lower bounds in challenging estimation problems and provides reproducible code for broader adoption in signal processing and related domains.

Abstract

The Bayesian Cramér-Rao bound (BCRB) is a crucial tool in signal processing for assessing the fundamental limitations of any estimation problem as well as benchmarking within a Bayesian frameworks. However, the BCRB cannot be computed without full knowledge of the prior and the measurement distributions. In this work, we propose a fully learned Bayesian Cramér-Rao bound (LBCRB) that learns both the prior and the measurement distributions. Specifically, we suggest two approaches to obtain the LBCRB: the Posterior Approach and the Measurement-Prior Approach. The Posterior Approach provides a simple method to obtain the LBCRB, whereas the Measurement-Prior Approach enables us to incorporate domain knowledge to improve the sample complexity and {interpretability}. To achieve this, we introduce a Physics-encoded score neural network which enables us to easily incorporate such domain knowledge into a neural network. We {study the learning} errors of the two suggested approaches theoretically, and validate them numerically. We demonstrate the two approaches on several signal processing examples, including a linear measurement problem with unknown mixing and Gaussian noise covariance matrices, frequency estimation, and quantized measurement. In addition, we test our approach on a nonlinear signal processing problem of frequency estimation with real-world underwater ambient noise.

Learned Bayesian Cramér-Rao Bound for Unknown Measurement Models Using Score Neural Networks

TL;DR

This work tackles computing a Bayesian Cramér-Rao bound (BCRB) when priors and measurement models are unknown by introducing two fully learned LBCRB frameworks. The Posterior Approach learns the posterior score and constructs the bound from data, while the Measurement-Prior Approach decomposes the bound into prior and Fisher-score components and integrates domain knowledge via a Physics-encoded Score Neural Network (PeSNN) and Fisher Score Matching. Theoretical results establish consistency and finite-sample error bounds for both methods, and experiments across linear and nonlinear problems (including quantization and underwater noise) demonstrate that the Measurement-Prior Approach often achieves lower sample complexity and higher accuracy, especially when physical models are available. The work enables practical, data-driven lower bounds in challenging estimation problems and provides reproducible code for broader adoption in signal processing and related domains.

Abstract

The Bayesian Cramér-Rao bound (BCRB) is a crucial tool in signal processing for assessing the fundamental limitations of any estimation problem as well as benchmarking within a Bayesian frameworks. However, the BCRB cannot be computed without full knowledge of the prior and the measurement distributions. In this work, we propose a fully learned Bayesian Cramér-Rao bound (LBCRB) that learns both the prior and the measurement distributions. Specifically, we suggest two approaches to obtain the LBCRB: the Posterior Approach and the Measurement-Prior Approach. The Posterior Approach provides a simple method to obtain the LBCRB, whereas the Measurement-Prior Approach enables us to incorporate domain knowledge to improve the sample complexity and {interpretability}. To achieve this, we introduce a Physics-encoded score neural network which enables us to easily incorporate such domain knowledge into a neural network. We {study the learning} errors of the two suggested approaches theoretically, and validate them numerically. We demonstrate the two approaches on several signal processing examples, including a linear measurement problem with unknown mixing and Gaussian noise covariance matrices, frequency estimation, and quantized measurement. In addition, we test our approach on a nonlinear signal processing problem of frequency estimation with real-world underwater ambient noise.

Paper Structure

This paper contains 66 sections, 24 theorems, 158 equations, 11 figures, 1 table.

Table of Contents

  1. Introduction
  2. Notation and Background
  3. Notations
  4. Bayesian Cramér Rao Bound
  5. Overview of Score Matching
  6. Learned Bayesian Cramér Rao Bound:Overview
  7. Posterior Approach
  8. Learning Step
  9. Evaluation Step
  10. Measurement-Prior Approach
  11. Learning Prior Step
  12. Construct Physics-encoded Score Neural Network (PeSNN)
  13. Learning Fisher Score Step
  14. Evaluation Step
  15. Derivation of the LBCRB
  16. ...and 51 more sections

Key Result

Theorem IV.1

Suppose that the boundary condition eq:boundary_condtions_gen_direct and the regularity conditions Assumptions ass:score_reg) hold. Define Then ${\mathcal{L}_{F}^{0}}={{\mathcal{L}}_{F}}+C$, where $C$ is a constant independent of $\Omega_{F}$.

Figures (11)

  • Figure 1: Physics-encoded Score Neural Network: The white blocks indicate the usage of $\mathcal{M}\left(\bm{\theta}\right)$ (a known function), whereas the green block represents a neural network that is trained during the learning process.
  • Figure 2: LBCRB: Overview. Fig. \ref{['fig:main_post']}: Posterior Approach. The learning phase consists of a single training step for the posterior score; this neural net model is subsequently used in the evaluation phase. Fig. \ref{['fig:main_lik_prior']}: Measurement-Prior Approach. The learning phase comprises two consecutive training steps: (i) training the prior score; and (ii) training the measurement score—yielding two distinct NN models: the Prior and the Fisher scores. These models are then applied in the evaluation phase.
  • Figure 3: Histograms (using 1000 Monte-Carlo trials) of the LBCRB relative errors $\mathrm{RE}_{B}$\ref{['eq:inv_bound_post']} and $\mathrm{RE}_{MP}$\ref{['eq:inv_bound_mp']} for a linear Gaussian example.
  • Figure 4: Empirical-mean errors $\mathrm{RE}_{B}^{(e)}$ and $\mathrm{RE}_{MP}^{(e)}$ vs. $N_{\mathcal{D}}$ for the linear Gaussian estimation problem with $m_{iid}=10$. Upper part: theoretical bounds \ref{['eq:re_error_post_mean']} and \ref{['eq:re_error_mp_mean']}. Bottom part: average and standard deviation of the actual empirical-mean error over 1000 Monte-Carlo trials.
  • Figure 5: Empirical-mean errors $\mathrm{RE}_{B}^{(e)}$ and $\mathrm{RE}_{MP}^{(e)}$ vs $m_{iid}$ for the linear Gaussian estimation problem. Upper part: theoretical bounds \ref{['eq:re_error_post_mean']} and \ref{['eq:re_error_mp_mean']}. Bottom part: average and standard deviation of the actual empirical-mean error over 1000 Monte-Carlo trials.
  • ...and 6 more figures

Theorems & Definitions (51)

  • Remark 1
  • Remark 2
  • Remark 3
  • Remark 4
  • Remark 5
  • Remark 6
  • Theorem IV.1: Fisher Score Matching
  • Remark 7
  • Definition V.1: Intrinsic Dimension of a Matrix
  • Theorem V.1: LBCRB Approximation Error: Posterior Approach
  • ...and 41 more