Design and Experimental Test of Datatic Approximate Optimal Filter in Nonlinear Dynamic Systems

TL;DR

The paper tackles state estimation in nonlinear, non-Gaussian dynamic systems where traditional Bayesian filters struggle. It formulates a Markovian Filtering Problem (MFP) and develops the datatic approximate optimal filter (DAOF), with two architectures: DAOF-v1 for systems with explicit models and DAOF-v2 for model-free operation, both trained via an actor-critic RL framework that minimizes the discounted accumulated error $\sum_{k=t}^{\infty} \gamma^{k-t} \phi(x_k, \hat{x}_k)$. Through experiments on 2-DOF and 14-DOF vehicle models with non-Gaussian noises, DAOF-v1 achieves superior accuracy and lower computation than UKF, PF, and SLF, while DAOF-v2 demonstrates robust model-free filtering capabilities for high-dimensional dynamics. The results indicate that a data-driven RL-based approach can yield practical, real-time state estimation in challenging nonlinear settings, broadening the applicability of datatic filtering to complex engineering systems.

Abstract

Filtering is crucial in engineering fields, providing vital state estimation for control systems. However, the nonlinear nature of complex systems and the presence of non-Gaussian noises pose significant challenges to the performance of conventional filtering methods in terms of estimation accuracy and computational efficiency. In this work, we present a data-driven closed-loop filter, termed datatic approximate optimal filter (DAOF), specifically designed for nonlinear systems under non-Gaussian conditions. We first formulate a Markovian filtering problem (MFP), which inherently shares a connection with reinforcement learning (RL) as it aims to compute the optimal state estimate by minimizing the accumulated error. To solve MFP, we propose DAOF, which primarily incorporates a trained RL policy and features two distinct structural designs: DAOF-v1 and DAOF-v2. Designed for systems with explicit models, DAOF-v1 combines prediction and update phases, with the RL policy generating the update value. Meanwhile, DAOF-v2 bypasses system modeling by directly outputting the state estimate. Then, we utilize an actor-critic algorithm to learn the parameterized policy for DAOF. Experimental results on a 2-degree-of-freedom (2-DOF) vehicle system, equipped with explicit system models, demonstrate the superior accuracy and computational efficiency of DAOF-v1 compared to existing nonlinear filters. Moreover, DAOF-v2 showcases its unique ability to perform filtering without requiring explicit system modeling, as validated by a 14-DOF vehicle system.

Figures (7)

• Figure 1: Hidden markov model for filtering problems.
• Figure 2: Training procedure of DAOF. The learned filter can be applied online.
• Figure 3: Error and state plot for experiment I.
• Figure 4: Box plot of RMSE for PF, SLF, DAOF-v1, and DAOF-v2, for experiment I.
• Figure 5: Comparison of training processes of SLF, DAOF-v1 and DAOF-v2 for experiment I.

Theorems & Definitions (4)

• Remark 1
• Remark 2
• Remark 3
• Remark 4