Language-Aided State Estimation

TL;DR

The paper tackles state estimation for physical systems when human observations arrive as natural language. It introduces the Language-Aided Particle Filter (LAPF), which processes texts $s_k$ through a quantized-label classifier to produce a distribution $p(q_k|s_k)$ and uses it to compute the likelihood $p(s_k|x_{k|k-1})$ in a particle filter, enabling human-in-the-loop estimation with $x_k \in \mathbb{R}^n$ and $s_k$ expressed via a cognitive model and a quantizer. The method is extended to multiple human sensors and demonstrated on a canal water-level estimation problem, showing that LAPF outperforms a Deep Neural Network–aided PF (EDAPF) and is more robust to out-of-domain language data. Key contributions include a tractable likelihood factorization across quantized labels and a robust framework for integrating general natural-language observations into PF-based state estimation, which can improve accuracy when physical sensors are sparse. The work has practical impact for incorporating widespread human-language signals into control and estimation tasks in real-world, sensor-limited environments.

Abstract

Natural language data, such as text and speech, have become readily available through social networking services and chat platforms. By leveraging human observations expressed in natural language, this paper addresses the problem of state estimation for physical systems, in which humans act as sensing agents. To this end, we propose a Language-Aided Particle Filter (LAPF), a particle filter framework that structures human observations via natural language processing and incorporates them into the update step of the state estimation. Finally, the LAPF is applied to the water level estimation problem in an irrigation canal and its effectiveness is demonstrated.

Figures (5)

• Figure 1: Human Sensor Observation Process (for $m=5$)
• Figure 2: Block Diagram of the Model for Computing the Probability Distribution $p (q_k | s_k)$
• Figure 3: Schematic of the Canal Setting with Observation Points
• Figure 4: Comparison of State-wise MSE: Observation-Free vs. EDAPF vs. LAPF
• Figure 5: Comparison of State-wise MSE: EDAPF vs. LAPF (Out-of-domain)