State Estimation Using Particle Filtering in Adaptive Machine Learning Methods: Integrating Q-Learning and NEAT Algorithms with Noisy Radar Measurements

TL;DR

This work tackles state estimation under sensor noise by integrating particle filtering with adaptive learning methods—Q-learning and NEAT—to produce reliable state estimates from noisy radar data. The approach refines observations into a posterior-like estimate that improves learning updates and controller evolution, leading to faster convergence, higher performance, and greater resilience to noise in both grid-based and car navigation tasks. Empirical results show significant gains in average rewards, stability, and success rates for Q-learning, and higher fitness, smoother policies, and better robustness for NEAT, compared to baselines without filtering. The study highlights state estimation as a critical component of adaptive learning systems in uncertain environments and points to diffusion-based and online parameter-filtering extensions as promising future directions.

Abstract

Reliable state estimation is essential for autonomous systems operating in complex, noisy environments. Classical filtering approaches, such as the Kalman filter, can struggle when facing nonlinear dynamics or non-Gaussian noise, and even more flexible particle filters often encounter sample degeneracy or high computational costs in large-scale domains. Meanwhile, adaptive machine learning techniques, including Q-learning and neuroevolutionary algorithms such as NEAT, rely heavily on accurate state feedback to guide learning; when sensor data are imperfect, these methods suffer from degraded convergence and suboptimal performance. In this paper, we propose an integrated framework that unifies particle filtering with Q-learning and NEAT to explicitly address the challenge of noisy measurements. By refining radar-based observations into reliable state estimates, our particle filter drives more stable policy updates (in Q-learning) or controller evolution (in NEAT), allowing both reinforcement learning and neuroevolution to converge faster, achieve higher returns or fitness, and exhibit greater resilience to sensor uncertainty. Experiments on grid-based navigation and a simulated car environment highlight consistent gains in training stability, final performance, and success rates over baselines lacking advanced filtering. Altogether, these findings underscore that accurate state estimation is not merely a preprocessing step, but a vital component capable of substantially enhancing adaptive machine learning in real-world applications plagued by sensor noise.

Figures (10)

• Figure 1: Average rewards with particle filtering.
• Figure 2: Average rewards without particle filtering.
• Figure 3: Average rewards (late phase) with upper/lower bounds, illustrating more stable performance under filtering.
• Figure 4: Smoothed average rewards (late phase). Particle filtering case converges more stably toward high returns.
• Figure 5: Near-optimal path (with filtering).