Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

State-Aware Timeliness in Energy Harvesting IoT Systems Monitoring a Markovian Source

Erfan Delfani, George J. Stamatakis, Nikolaos Pappas

TL;DR

This work tackles timely status updates in an energy-harvesting IoT system monitoring a two-state Markov source with state-dependent demand for freshness. It models the problem as a finite, infinite-horizon MDP with per-state AoI variables $Δ^0$ and $Δ^1$ and a per-step cost $g(s,a)=(1-Z)Δ^0 + Z(Δ^1)^2$, solved via a contraction Bellman operator and Value Iteration to obtain a stationary policy. A key theoretical contribution is showing that the optimal policy is a two-dimensional threshold policy in the AoI variables, and numerical results illustrate how energy buffer capacity, harvest/transmission probabilities, and source-transition dynamics shape the optimal thresholds and overall cost. The findings provide a principled framework for state-aware freshness in EH IoT systems and point to scalable extensions using DRL when the model is unknown or the state space is large.

Abstract

In this study, we investigate the optimal transmission policies within an energy harvesting status update system, where the demand for status updates depends on the state of the source. The system monitors a two-state Markovian source that characterizes a stochastic process, which can be in either a normal state or an alarm state, with a higher demand for fresh updates when the source is in the alarm state. We propose a metric to capture the freshness of status updates for each state of the stochastic process by introducing two Age of Information (AoI) variables, extending the definition of AoI to account for the state changes of the stochastic process. We formulate the problem as a Markov Decision Process (MDP), utilizing a transition cost function that applies linear and non-linear penalties based on AoI and the state of the stochastic process. Through analytical investigation, we delve into the structure of the optimal transmission policy for the resulting MDP problem. Furthermore, we evaluate the derived policies via numerical results and demonstrate their effectiveness in reserving energy in anticipation of forthcoming alarm states.

State-Aware Timeliness in Energy Harvesting IoT Systems Monitoring a Markovian Source

TL;DR

This work tackles timely status updates in an energy-harvesting IoT system monitoring a two-state Markov source with state-dependent demand for freshness. It models the problem as a finite, infinite-horizon MDP with per-state AoI variables and and a per-step cost , solved via a contraction Bellman operator and Value Iteration to obtain a stationary policy. A key theoretical contribution is showing that the optimal policy is a two-dimensional threshold policy in the AoI variables, and numerical results illustrate how energy buffer capacity, harvest/transmission probabilities, and source-transition dynamics shape the optimal thresholds and overall cost. The findings provide a principled framework for state-aware freshness in EH IoT systems and point to scalable extensions using DRL when the model is unknown or the state space is large.

Abstract

In this study, we investigate the optimal transmission policies within an energy harvesting status update system, where the demand for status updates depends on the state of the source. The system monitors a two-state Markovian source that characterizes a stochastic process, which can be in either a normal state or an alarm state, with a higher demand for fresh updates when the source is in the alarm state. We propose a metric to capture the freshness of status updates for each state of the stochastic process by introducing two Age of Information (AoI) variables, extending the definition of AoI to account for the state changes of the stochastic process. We formulate the problem as a Markov Decision Process (MDP), utilizing a transition cost function that applies linear and non-linear penalties based on AoI and the state of the stochastic process. Through analytical investigation, we delve into the structure of the optimal transmission policy for the resulting MDP problem. Furthermore, we evaluate the derived policies via numerical results and demonstrate their effectiveness in reserving energy in anticipation of forthcoming alarm states.
Paper Structure (10 sections, 2 theorems, 81 equations, 7 figures)

This paper contains 10 sections, 2 theorems, 81 equations, 7 figures.

Table of Contents

  1. Introduction
  2. System Model
  3. Problem Formulation
  4. Analytical Results
  5. Optimal Policy
  6. Threshold Policy
  7. Numerical Results
  8. Conclusions and Future work
  9. Proof of Theorem 1
  10. Lemma 1

Key Result

Theorem 1

An optimal policy of the MDP problem is a threshold policy.

Figures (7)

  • Figure 1: An EH status update system for a stochastic process with normal and alarm states.
  • Figure 2: The first sub-figure presents the time evolution of the stochastic process' state. The second sub-figure presents the evolution of the AoI for each state of the stochastic process.
  • Figure 3: Impact of energy buffer's capacity, $E_{max}$, on $J^*(s_0)$.
  • Figure 4: Impact of transmission success probability $P_s$ on $J^*(s_0)$.
  • Figure 5: Impact of different stochastic process's state transition probabilities on $J^*(s_0)$.
  • ...and 2 more figures

Theorems & Definitions (6)

  • Definition 1
  • Theorem 1
  • proof
  • proof
  • Lemma 1
  • proof