Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Attention-Driven Hierarchical Reinforcement Learning with Particle Filtering for Source Localization in Dynamic Fields

Yiwei Shi, Mengyue Yang, Qi Zhang, Weinan Zhang, Cunjia Liu, Weiru Liu

TL;DR

The paper tackles inverse source localization and characterization (ISLC) in dynamic, partially observable fields by introducing a hierarchical framework that blends attention-guided Bayesian inference with reinforcement learning. It leverages an attention-enhanced particle filter to maintain a belief over source parameters and supports two execution modes: ATT-PFP for planning and ATT-PFRL for real-time RL, all within a POMDP setting. A convergence analysis for the attention-augmented filter is provided, and extensive experiments show improved accuracy, adaptability, and computational efficiency, including out-of-distribution scenarios. Overall, the work advances dynamic field estimation by focusing attention on informative belief regions and integrating planning and learning for robust decision-making under uncertainty.

Abstract

In many real-world scenarios, such as gas leak detection or environmental pollutant tracking, solving the Inverse Source Localization and Characterization problem involves navigating complex, dynamic fields with sparse and noisy observations. Traditional methods face significant challenges, including partial observability, temporal and spatial dynamics, out-of-distribution generalization, and reward sparsity. To address these issues, we propose a hierarchical framework that integrates Bayesian inference and reinforcement learning. The framework leverages an attention-enhanced particle filtering mechanism for efficient and accurate belief updates, and incorporates two complementary execution strategies: Attention Particle Filtering Planning and Attention Particle Filtering Reinforcement Learning. These approaches optimize exploration and adaptation under uncertainty. Theoretical analysis proves the convergence of the attention-enhanced particle filter, while extensive experiments across diverse scenarios validate the framework's superior accuracy, adaptability, and computational efficiency. Our results highlight the framework's potential for broad applications in dynamic field estimation tasks.

Attention-Driven Hierarchical Reinforcement Learning with Particle Filtering for Source Localization in Dynamic Fields

TL;DR

The paper tackles inverse source localization and characterization (ISLC) in dynamic, partially observable fields by introducing a hierarchical framework that blends attention-guided Bayesian inference with reinforcement learning. It leverages an attention-enhanced particle filter to maintain a belief over source parameters and supports two execution modes: ATT-PFP for planning and ATT-PFRL for real-time RL, all within a POMDP setting. A convergence analysis for the attention-augmented filter is provided, and extensive experiments show improved accuracy, adaptability, and computational efficiency, including out-of-distribution scenarios. Overall, the work advances dynamic field estimation by focusing attention on informative belief regions and integrating planning and learning for robust decision-making under uncertainty.

Abstract

In many real-world scenarios, such as gas leak detection or environmental pollutant tracking, solving the Inverse Source Localization and Characterization problem involves navigating complex, dynamic fields with sparse and noisy observations. Traditional methods face significant challenges, including partial observability, temporal and spatial dynamics, out-of-distribution generalization, and reward sparsity. To address these issues, we propose a hierarchical framework that integrates Bayesian inference and reinforcement learning. The framework leverages an attention-enhanced particle filtering mechanism for efficient and accurate belief updates, and incorporates two complementary execution strategies: Attention Particle Filtering Planning and Attention Particle Filtering Reinforcement Learning. These approaches optimize exploration and adaptation under uncertainty. Theoretical analysis proves the convergence of the attention-enhanced particle filter, while extensive experiments across diverse scenarios validate the framework's superior accuracy, adaptability, and computational efficiency. Our results highlight the framework's potential for broad applications in dynamic field estimation tasks.
Paper Structure (28 sections, 1 theorem, 17 equations, 4 figures, 3 tables)

This paper contains 28 sections, 1 theorem, 17 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Peliminaries
  4. Unified Mathematical Framework for Field Model Based on the Convection-Diffusion Equation
  5. Gaussian Plume Model
  6. Partially Observable Markov Decision Process
  7. Sequential Importance Sampling Convergence Theorem
  8. Methodology
  9. Bayesian Approximation for Belief Distribution
  10. Particle Filter for Iterative Distribution Approximation.
  11. Sequential Importance Sampling (SIS).
  12. Simplification under Fixed Parameters.
  13. Convergence Properties.
  14. MCMC Process with Attention Mechanism
  15. Effective Sample Size (ESS) and Resampling
  16. ...and 13 more sections

Key Result

Theorem 3.1

Let $\Theta_k$ represent the latent state at time $k$, and let $o_{1:k} = \{o_1, \dots, o_k\}$ denote the sequence of observations. Assume a Bayesian model: where $p(\Theta_k)$ is the prior and $p(o_{1:k} | \Theta_k)$ the likelihood. Let $\hat{p}_N^{(\text{SIS})}(\Theta_k | o_{1:k})$ be the particle-based approximation of the posterior. Then, under standard assumptions (e.g., bounded likelihood,

Figures (4)

  • Figure 1: Hierarchical Attention-Based Inference Framework
  • Figure 2: Gaussian Plume Mode Visualisation
  • Figure 3: Design of Experiments
  • Figure 4: Out of Distribution (left) and ablation study(right)

Theorems & Definitions (1)

  • Theorem 3.1: Convergence of Standard SIS doucet2001sequentialdoucet2009tutorial