Real-time Tracking in a Status Update System with an Imperfect Feedback Channel

TL;DR

The paper tackles real-time tracking with an energy-harvesting transmitter and imperfect forward/feedback channels by formulating the problem as a POMDP and converting it to a finite-state belief-MDP solved via relative value iteration. To manage complexity, it introduces a belief-space truncation and develops two low-complexity per-slot policies: an energy-agnostic LC policy and an energy-aware LC policy that uses a regularization term to conserve energy. Numerical results demonstrate threshold/switching structures in the optimal and sub-optimal policies and show that the energy-aware LC policy closely matches the POMDP-based policy, especially under limited energy availability. The work provides practical strategies for minimizing a generic distortion in EH status-update systems with imperfect feedback, with implications for low-resource IoT and vehicular networks. It uses a distortion metric $d(X,\hat{X})$ and demonstrates how channel reliability, energy arrivals, and source dynamics shape transmission decisions.

Abstract

We consider a status update system consisting of a finite-state Markov source, an energy-harvesting-enabled transmitter, and a sink. The forward and feedback channels between the transmitter and the sink are error-prone. We study the problem of minimizing the long-term time average of a (generic) distortion function subject to an energy causality constraint. Since the feedback channel is error-prone, the transmitter has only partial knowledge about the transmission results and, consequently, about the estimate of the source state at the sink. Therefore, we model the problem as a partially observable Markov decision process (POMDP), which is then cast as a belief-MDP problem. The infinite belief space makes solving the belief-MDP difficult. Thus, by exploiting a specific property of the belief evolution, we truncate the state space and formulate a finite-state MDP problem, which is then solved using the relative value iteration algorithm (RVIA). Furthermore, we propose a low-complexity transmission policy in which the belief-MDP problem is transformed into a sequence of per-slot optimization problems. Simulation results show the effectiveness of the proposed policies and their superiority compared to a baseline policy. Moreover, we numerically show that the proposed policies have switching-type structures.

Figures (11)

• Figure 1: The system model with the considered source model.
• Figure 2: The average cost $\bar{C}$ versus $m$ for different reliabilities of the forward and feedback channels where the source is characterized as $p = 0.7$, $N = 3$ and the EH module is characterized as $\mu = 0.5$, $B = 3$.
• Figure 3: The (instantaneous) average cost $\bar{C}$ for different values of successful reception in the forward channel for the POMDP-based policy with respect to time slots where $p = 0.7$, $N = 3$, $\mu = 0.5$, $B = 3$, and $p_f = 0.7$.
• Figure 4: Structure of the POMDP-based policy for a binary source with respect to the belief and energy level of the battery $b$ for three different energy arrival rate $\mu$, where $X=1$, $p = 0.7$, $N = 2$, channel reliabilities are defined as $p_s=0.6$, $p_f = 0.7$, and $B = 3$.
• Figure 5: Structure of the POMDP-based policy for a binary source with respect to the belief and energy level of the battery $b$ for three different source's self-transition probability $p$, where $X=1$, $N = 2$, $p_s=0.4$, $p_f = 0.5$, $\mu = 0.7$, and $B = 3$.

Theorems & Definitions (8)

• Proposition 1
• proof
• Theorem 1
• proof
• Corollary 1.1
• proof
• Corollary 1.2: Bellman's Equation
• proof