Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Credible Uncertainty Quantification under Noise and System Model Mismatch

Penggao Ya, Li-Ta Hsu, Rui Sun

TL;DR

A unified, multi-metric framework that integrates noncredibility index, negative log-likelihood, and energy score metrics, featuring an empirical location test (ELT) to detect system model bias and a directional probing technique that uses the metrics'asymmetric sensitivities to distinguish NMM from SMM is developed.

Abstract

State estimators often provide self-assessed uncertainty metrics, such as covariance matrices, whose credibility is critical for downstream tasks. However, these self-assessments can be misleading due to underlying modeling violations like noise model mismatch (NMM) or system model misspecification (SMM). This letter addresses this problem by developing a unified, multi-metric framework that integrates noncredibility index (NCI), negative log-likelihood (NLL), and energy score (ES) metrics, featuring an empirical location test (ELT) to detect system model bias and a directional probing technique that uses the metrics' asymmetric sensitivities to distinguish NMM from SMM. Monte Carlo simulations reveal that the proposed method achieves excellent diagnosis accuracy (80-100%) and significantly outperforms single-metric diagnosis methods. The effectiveness of the proposed method is further validated on a real-world UWB positioning dataset. This framework provides a practical tool for turning patterns of credibility indicators into actionable diagnoses of model deficiencies.

Credible Uncertainty Quantification under Noise and System Model Mismatch

TL;DR

A unified, multi-metric framework that integrates noncredibility index, negative log-likelihood, and energy score metrics, featuring an empirical location test (ELT) to detect system model bias and a directional probing technique that uses the metrics'asymmetric sensitivities to distinguish NMM from SMM is developed.

Abstract

State estimators often provide self-assessed uncertainty metrics, such as covariance matrices, whose credibility is critical for downstream tasks. However, these self-assessments can be misleading due to underlying modeling violations like noise model mismatch (NMM) or system model misspecification (SMM). This letter addresses this problem by developing a unified, multi-metric framework that integrates noncredibility index (NCI), negative log-likelihood (NLL), and energy score (ES) metrics, featuring an empirical location test (ELT) to detect system model bias and a directional probing technique that uses the metrics' asymmetric sensitivities to distinguish NMM from SMM. Monte Carlo simulations reveal that the proposed method achieves excellent diagnosis accuracy (80-100%) and significantly outperforms single-metric diagnosis methods. The effectiveness of the proposed method is further validated on a real-world UWB positioning dataset. This framework provides a practical tool for turning patterns of credibility indicators into actionable diagnoses of model deficiencies.

Paper Structure

This paper contains 18 sections, 9 equations, 2 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Credibility Metrics
  3. Normalized Estimation Error Squared (NEES)
  4. Noncredibility index (NCI)
  5. Negative Log-Likelihood (NLL)
  6. Energy Score (ES)
  7. Unified Credibility Diagnosis Scheme
  8. Identify the impacts of SMM
  9. Evaluation of noncredibility direction without SMM
  10. Evaluation of noncredibility direction with SMM
  11. Reduce the impacts of SMM
  12. Use NCI to tentatively evaluate pessimism
  13. Directional probes using NLL and ES
  14. Simulation Experiments
  15. Experimental Design
  16. ...and 3 more sections

Figures (2)

  • Figure 1: The flowchart of the proposed unified credibility diagnosis method.
  • Figure 2: The 2-D trjactory of the UWB tag and positioning solutions at six static locations. The 'star' stands for ground-truth location and the 'circle' represents the positioning estimation.