Real-Time Performance Analysis of Multi-Fidelity Residual Physics-Informed Neural Process-Based State Estimation for Robotic Systems

TL;DR

The paper addresses real-time state estimation for robotic systems under model mismatch and sensor noise by introducing the Multi-Fidelity Residual Physics-Informed Neural Process (MFR-PINP). The method fuses low-fidelity physics priors with high-fidelity residual learning within a neural-process framework and augments uncertainty with split conformal prediction, enabling calibrated, real-time predictions. Trained with dual ELBO objectives in an online, hybrid setting, MFR-PINP is validated on a GPS-denied skid-steering robot and outperforms UKF and transformer-based DKF baselines in RMSE and distributional accuracy (NLL), with sharper uncertainty bounds thanks to conformal calibration. The work demonstrates practical real-time applicability at 50 Hz and highlights potential for safer, uncertainty-aware robotics in challenging environments, with future directions including data-efficient training and broader deployment in multi-agent and control contexts.

Abstract

Various neural network architectures are used in many of the state-of-the-art approaches for real-time nonlinear state estimation. With the ever-increasing incorporation of these data-driven models into the estimation domain, model predictions with reliable margins of error are a requirement -- especially for safety-critical applications. This paper discusses the application of a novel real-time, data-driven estimation approach based on the multi-fidelity residual physics-informed neural process (MFR-PINP) toward the real-time state estimation of a robotic system. Specifically, we address the model-mismatch issue of selecting an accurate kinematic model by tasking the MFR-PINP to also learn the residuals between simple, low-fidelity predictions and complex, high-fidelity ground-truth dynamics. To account for model uncertainty present in a physical implementation, robust uncertainty guarantees from the split conformal (SC) prediction framework are modeled in the training and inference paradigms. We provide implementation details of our MFR-PINP-based estimator for a hybrid online learning setting to validate our model's usage in real-time applications. Experimental results of our approach's performance in comparison to the state-of-the-art variants of the Kalman filter (i.e. unscented Kalman filter and deep Kalman filter) in estimation scenarios showed promising results for the MFR-PINP model as a viable option in real-time estimation tasks.

Figures (5)

• Figure 1: Architecture and Direct Probabilistic Graph of MFR-PINP model where arrows show both model flow and conditional dependence. We display the notation for context $\phi_{C}^{\text{low}},\phi_{C}^{\mathcal{R}}$ and target $\phi_{T}^{\text{low}},\phi_{T}^{\mathcal{R}}$ sets utilized by both NP models. Note that shaded circles denote observed parameters, white denote learned parameters, blue denote utilized neural networks, and green represents physics-informed priors that are given to model decoders as input. We explicitly state that our model formulation builds on model structure discussed in niu2024multi by incorporating physics-informed priors in model decoders for improved performance in modeling key conditional distributions $p(y_{T}^{\text{low}}|x_{T}^{\text{low}},z^{\text{low}},\hat{y}_{T}^{\text{low}},A)\;\;\text{and}\;\;p(r_{k+1}|x_{T}^{\text{high}},z^{\text{high}},\hat{r}_{k+1})$.
• Figure 2: Four-wheeled robotic platform used in study
• Figure 3: Model training dynamics of the MFR-PINP and transformer-based deep Kalman filter (DKF) goel2024can
• Figure 4: Distribution error dynamics of the split conformal method-augmented MFR-PINP (SC calibrated), the non-split conformal MFR-PINP (uncalibrated), and the deep Kalman filter (DKF)
• Figure 5: Visual results on ground-truth test trajectory (green) from MFR-PINP (blue) and DKF goel2024can (red).