Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Minimizing Age of Detection for a Markov Source over a Lossy Channel

Shivang Garde, Jaya Prakash Champati, Arpan Chattopadhyay

TL;DR

Problem: timely detection of DTMC state transitions via a pull-based sensor over a lossy wireless channel. Approach: define AoD as a semantics-aware freshness metric and minimize its long-term average under a sampling-frequency constraint by formulating a Constrained Markov Decision Problem (CMDP) and solving the Lagrangian MDP with Relative Value Iteration. Key contributions: (i) formal AoD definition extending age-based metrics; (ii) CMDP formulation and solution via a Lagrangian MDP; (iii) numerical results showing AoD decays with transmission success probability $q$ and yields lower estimation error than AoI under various settings. Significance: provides a practical, improvement-oriented freshness metric for remote monitoring of Markov sources in unreliable channels, enabling energy- and bandwidth-efficient sampling policies.

Abstract

Monitoring a process/phenomenon of specific interest is prevalent in Cyber-Physical Systems (CPS), remote healthcare, smart buildings, intelligent transport, industry 4.0, etc. A key building block of the monitoring system is a sensor sampling the process and communicating the status updates to a monitor for detecting events of interest. Measuring the freshness of the status updates is essential for the timely detection of events, and it has received significant research interest in recent times. In this paper, we propose a new freshness metric, Age of Detection (AoD), for monitoring the state transitions of a Discrete Time Markov Chain (DTMC) source over a lossy wireless channel. We consider the pull model where the sensor samples DTMC state whenever the monitor requests a status update. We formulate a Constrained Markov Decision Problem (CMDP) for optimising the AoD subject to a constraint on the average sampling frequency and solve it using the Lagrangian MDP formulation and Relative Value Iteration (RVI) algorithm. Our numerical results show interesting trade-offs between AoD, sampling frequency, and transmission success probability. Further, the AoD minimizing policy provides a lower estimation error than the Age of Information (AoI) minimizing policy, thus demonstrating the utility of AoD for monitoring DTMC sources.

Minimizing Age of Detection for a Markov Source over a Lossy Channel

TL;DR

Problem: timely detection of DTMC state transitions via a pull-based sensor over a lossy wireless channel. Approach: define AoD as a semantics-aware freshness metric and minimize its long-term average under a sampling-frequency constraint by formulating a Constrained Markov Decision Problem (CMDP) and solving the Lagrangian MDP with Relative Value Iteration. Key contributions: (i) formal AoD definition extending age-based metrics; (ii) CMDP formulation and solution via a Lagrangian MDP; (iii) numerical results showing AoD decays with transmission success probability and yields lower estimation error than AoI under various settings. Significance: provides a practical, improvement-oriented freshness metric for remote monitoring of Markov sources in unreliable channels, enabling energy- and bandwidth-efficient sampling policies.

Abstract

Monitoring a process/phenomenon of specific interest is prevalent in Cyber-Physical Systems (CPS), remote healthcare, smart buildings, intelligent transport, industry 4.0, etc. A key building block of the monitoring system is a sensor sampling the process and communicating the status updates to a monitor for detecting events of interest. Measuring the freshness of the status updates is essential for the timely detection of events, and it has received significant research interest in recent times. In this paper, we propose a new freshness metric, Age of Detection (AoD), for monitoring the state transitions of a Discrete Time Markov Chain (DTMC) source over a lossy wireless channel. We consider the pull model where the sensor samples DTMC state whenever the monitor requests a status update. We formulate a Constrained Markov Decision Problem (CMDP) for optimising the AoD subject to a constraint on the average sampling frequency and solve it using the Lagrangian MDP formulation and Relative Value Iteration (RVI) algorithm. Our numerical results show interesting trade-offs between AoD, sampling frequency, and transmission success probability. Further, the AoD minimizing policy provides a lower estimation error than the Age of Information (AoI) minimizing policy, thus demonstrating the utility of AoD for monitoring DTMC sources.

Paper Structure

This paper contains 14 sections, 1 theorem, 11 equations, 10 figures.

Table of Contents

  1. Introduction
  2. Age of Detection: Definition and Analysis
  3. System Model
  4. Elements of MDP
  5. State space
  6. Action space
  7. Transition Probabilities
  8. AoD
  9. Lagrangian MDP
  10. Numerical Results
  11. Variation with transition probability
  12. Variation with transmission success probability $q$
  13. Comparison of estimation error under AoD and AoI
  14. Conclusion

Key Result

Proposition 1

If the wireless channel has no loss, i.e., $q=1$, then the average AoD equals the average age penalty.

Figures (10)

  • Figure 1: The pull model used for sampling information for a DTMC source's state transitions.
  • Figure 2: A pictorial representation showing the sample generation and reception instances over a short period of time.
  • Figure 3: Division of states on the basis of optimal decision for each state for $p_{01}=0.02$, $p_{10}=0.01$, $q = 0.8$ and $\nu=0.1$.
  • Figure 4: Division of states on the basis of optimal decision for each state for $p_{01}=0.03$, $p_{10}=0.01$, $q = 0.8$ and $\nu=0.1$.
  • Figure 5: Division of states on the basis of optimal decision for each state for $p_{01}=0.04$, $p_{10}=0.01$, $q = 0.8$ and $\nu=0.1$.
  • ...and 5 more figures

Theorems & Definitions (2)

  • Proposition 1
  • proof