Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Remote Tracking with State-Dependent Sensing in Pull-Based Systems: A POMDP Framework

Jiapei Tian, Abolfazl Zakeri, Marian Codreanu, David Gundlegård

TL;DR

A truncation-based approximation is developed that yields a finite-state MDP solved via the relative value iteration algorithm (RVIA) and a switching-type structure of the RVIA-based policy over the belief simplex is revealed, highlighting the importance of accounting for state-dependent sensing.

Abstract

We consider real-time remote tracking of a Markov source observed by multiple heterogeneous sensors with state-dependent sensing accuracy, motivated by distributed camera networks with overlapping coverage and spatial blind spots. Upon commands from a remote sink, sensors transmit their observations over error-prone channels. We aim to minimize the long-term average of a weighted sum of goal-aware distortion and transmission costs. The problem is formulated as a partially observable Markov decision process (POMDP) and cast into an equivalent belief-MDP. To address the intractability of the infinite and continuous belief space, we develop a truncation-based approximation that yields a finite-state MDP solved via the relative value iteration algorithm (RVIA). We further reformulate the original belief-MDP into a discounted version and solve it using incremental pruning algorithm (IPA). Numerical results demonstrate that the performance of the RVIA-based policy improves with the truncation depth at the expense of computational effort, and both proposed methods outperform low-complexity baselines across a wide range of system parameters. The results also reveal a switching-type structure of the RVIA-based policy over the belief simplex and quantify the impact of key system parameters, highlighting the importance of accounting for state-dependent sensing.

Remote Tracking with State-Dependent Sensing in Pull-Based Systems: A POMDP Framework

TL;DR

A truncation-based approximation is developed that yields a finite-state MDP solved via the relative value iteration algorithm (RVIA) and a switching-type structure of the RVIA-based policy over the belief simplex is revealed, highlighting the importance of accounting for state-dependent sensing.

Abstract

We consider real-time remote tracking of a Markov source observed by multiple heterogeneous sensors with state-dependent sensing accuracy, motivated by distributed camera networks with overlapping coverage and spatial blind spots. Upon commands from a remote sink, sensors transmit their observations over error-prone channels. We aim to minimize the long-term average of a weighted sum of goal-aware distortion and transmission costs. The problem is formulated as a partially observable Markov decision process (POMDP) and cast into an equivalent belief-MDP. To address the intractability of the infinite and continuous belief space, we develop a truncation-based approximation that yields a finite-state MDP solved via the relative value iteration algorithm (RVIA). We further reformulate the original belief-MDP into a discounted version and solve it using incremental pruning algorithm (IPA). Numerical results demonstrate that the performance of the RVIA-based policy improves with the truncation depth at the expense of computational effort, and both proposed methods outperform low-complexity baselines across a wide range of system parameters. The results also reveal a switching-type structure of the RVIA-based policy over the belief simplex and quantify the impact of key system parameters, highlighting the importance of accounting for state-dependent sensing.

Paper Structure

This paper contains 17 sections, 3 theorems, 45 equations, 9 figures, 2 algorithms.

Table of Contents

  1. Introduction
  2. Contributions
  3. Notations and Paper Organization
  4. Related Work
  5. System Model and Problem Formulation
  6. Proposed solutions
  7. POMDP Formualtion
  8. Belief-MDP Reformulation
  9. Solution via Belief Space Truncation
  10. Solution Via Incremental Pruning
  11. Low-Complexity Methods
  12. Numerical Results
  13. Simulation Setup
  14. Performance Evaluation
  15. CONCLUSIONS
  16. ...and 2 more sections

Key Result

Proposition 1

The belief $\mathbf{b}_t$ at slot $t$ is a sufficient statistic for the complete information state $I_{t}$, i.e., there exists a belief update function $\tau$ such that and belief update (eq:belief_update_function) can be expressed in terms of observation function (Eq:observation_function) and source transition probability matrix $\mathbf{P}$ as where $\mathbf{U}(a_t,o_{t+1})$ is a diagonal matr

Figures (9)

  • Figure 1: System model
  • Figure 2: Illustration of multi-sensor tracking scenario with overlapping area for different detection decay factors $\xi$.
  • Figure 3: Long-term average cost against window size $K$ for different channel reliabilities.
  • Figure 4: Long-term average cost for different policies with respect to time slots for $K=7,\ p=0.7,\ q=0.8$.
  • Figure 5: Long-term average cost and entropy rate against source self-transition probability $p$ for channel reliability $q=0.8$, transmission coefficient $\alpha=0.3$, source state transition parameters $\beta=1$.
  • ...and 4 more figures

Theorems & Definitions (3)

  • Proposition 1
  • Corollary 1
  • Proposition 2