Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Age of Incorrect Information for Generic Discrete-Time Markov Sources

Konstantinos Bountrogiannis, Anthony Ephremides, Panagiotis Tsakalides, George Tzagkarakis

Abstract

This work introduces a framework for analyzing the Age of Incorrect Information (AoII) in a real-time monitoring system with a generic discrete-time Markov source. We study a noisy communication system employing a hybrid automatic repeat request (HARQ) protocol, subject to a transmission rate constraint. The optimization problem is formulated as a constrained Markov decision process (CMDP), and it is shown that there exists an optimal policy that is a randomized mixture of two stationary policies. To overcome the intractability of computing the optimal stationary policies, we develop a multiple-threshold policy class where thresholds depend on the source, the receiver, and the packet count. By establishing a Markov renewal structure induced by threshold policies, we derive closed-form expressions for the long-term average AoII and transmission rate. The proposed policy is constructed via a relative value iteration algorithm that leverages the threshold structure to skip computations, combined with a bisection search to satisfy the rate constraint. To accommodate scenarios requiring lower computational complexity, we adapt the same technique to produce a simpler single-threshold policy that trades optimality for efficiency. Numerical experiments exhibit that both thresholdbased policies outperform periodic scheduling, with the multiplethreshold approach matching the performance of the globally optimal policy.

Age of Incorrect Information for Generic Discrete-Time Markov Sources

Abstract

This work introduces a framework for analyzing the Age of Incorrect Information (AoII) in a real-time monitoring system with a generic discrete-time Markov source. We study a noisy communication system employing a hybrid automatic repeat request (HARQ) protocol, subject to a transmission rate constraint. The optimization problem is formulated as a constrained Markov decision process (CMDP), and it is shown that there exists an optimal policy that is a randomized mixture of two stationary policies. To overcome the intractability of computing the optimal stationary policies, we develop a multiple-threshold policy class where thresholds depend on the source, the receiver, and the packet count. By establishing a Markov renewal structure induced by threshold policies, we derive closed-form expressions for the long-term average AoII and transmission rate. The proposed policy is constructed via a relative value iteration algorithm that leverages the threshold structure to skip computations, combined with a bisection search to satisfy the rate constraint. To accommodate scenarios requiring lower computational complexity, we adapt the same technique to produce a simpler single-threshold policy that trades optimality for efficiency. Numerical experiments exhibit that both thresholdbased policies outperform periodic scheduling, with the multiplethreshold approach matching the performance of the globally optimal policy.

Paper Structure

This paper contains 17 sections, 8 theorems, 59 equations, 5 figures, 1 table, 3 algorithms.

Table of Contents

  1. Introduction
  2. Problem Definition
  3. Communication Model
  4. Optimization Problem
  5. Optimal Policy Structure
  6. F(S,W,r): The Class of Full Space Multiple-Threshold Policies
  7. Analysis of F(S,W,r)
  8. Approximation of F(S,W,r)
  9. Approximation of the Optimal Policy
  10. F(1): The Class of Single-Threshold Policies
  11. Numerical Results
  12. Conclusion
  13. Proof of Theorem \ref{['thm:cmdp_policy']}
  14. Proof of Theorem \ref{['thm:param']}
  15. Proof of Theorem \ref{['thm:param']}
  16. ...and 2 more sections

Key Result

Theorem 1

There exists an optimal policy $\psi^*$ of eq:main_opt_problem, which is a randomized mixture of two stationary policies $\psi_{\lambda^+}$ and $\psi_{\lambda^-}$ that are both optimal solutions of the Lagrangian MDP eq:lagrange_problem with parameters $\lambda^+$ and $\lambda^-$, respectively. In p where $\bar{R}_{\psi_{\lambda}}$ is the transmission rate achieved by $\psi$. The optimal policy $\

Figures (5)

  • Figure 1: Concurrent discrete-time sample paths of the AoI and AoII. A binary source is sampled at slots $s_i$, $i=1,2$, and decoded by the receiver at slots $d_i$, $i=1,2$, respectively. The distortion is the absolute difference between the source and monitor values.
  • Figure 2: A generic Markov chain of $N=4$ states.
  • Figure 3: $\psi^*_\lambda$ policy (wait, transmit) for every source-receiver $(S,W)$ pair, and packet count $r=1$ (top) and $r=2$ (bottom). Note: When $S=W$, then $\delta=0$ always. The maximum transmission packet count is $r_\text{max}=2$, and the probability of successful decoding $d(r)$ is $0.5$ and $0.75$ for $r=1$ and $r=2$, respectively. The transmission penalty $\lambda=8$, while the source transition probabilities $p_{i,j}$ are given by the transition matrix $\left[ \mathstrut 0.520.120.180.18 \mathstrut0.170.570.170.09 \mathstrut0.030.060.720.19 \mathstrut0.160.100.180.56 \mathstrut \right]$.
  • Figure 4: The stationary threshold policy component $\psi_{\lambda^+}$ of the randomized policy $\psi^*$ for each of the threshold policy classes introduced in the previous sections. For the policy in $\mathcal{F}(S,W,r)$, only the thresholds for packet count $r=1$ are shown. The transmission rate is $R=0.1$, the maximum transmission packet count is $r_\text{max}=2$, and the probability of successful decoding $d(r)$ is $0.5$ and $0.75$ for $r=1$ and $r=2$, respectively. The source transition probabilities $p_{i,j}$ are given by the transition matrix $\left[ \mathstrut 0.520.120.180.18 \mathstrut0.170.570.170.09 \mathstrut0.030.060.720.19 \mathstrut0.160.100.180.56 \mathstrut \right]$.
  • Figure 5: Average AoII versus transmission rate for a random source - $N=4$

Theorems & Definitions (29)

  • Definition 1: Main Optimization Problem
  • Definition 2: Lagrangian Problem
  • Theorem 1
  • proof
  • Proposition 1
  • proof
  • Definition 3
  • Corollary 1
  • Corollary 2
  • Theorem 2
  • ...and 19 more