Multi-Threshold AoII-Optimum Sampling Policies for CTMC Information Sources

TL;DR

The paper addresses minimizing AoII in push-based remote estimation of finite-state CTMC sources over a delay channel under a sampling-budget constraint. It introduces the MRPH framework to model time-to-absorption with piecewise-constant CTMC generators and formulates an infinite-horizon CSMDP, solved via Lagrangian relaxation and policy iteration to show ESAT is optimal with multiple thresholds. To reduce complexity, it proposes two sub-optimal policies, EAT and ST, with tractable threshold computations and guarantees optimality for special cases (binary and symmetric CTMCs). Numerical results across symmetric, binary, ternary, and larger-state sources validate MRPH-based analysis and demonstrate AoII gains versus Poisson baselines. Overall, MRPH enables exact AoII analysis for multi-threshold policies, providing scalable strategies for practical CTMC-based freshness optimization.

Abstract

We study push-based sampling and transmission policies for a status update system consisting of a general finite-state continuous-time Markov chain (CTMC) information source with known dynamics, with the goal of minimizing the average age of incorrect information (AoII). The problem setting we investigate involves an exponentially distributed delay channel for transmissions and a constraint on the average sampling rate. We first show that the optimum sampling and transmission policy is a 'multi-threshold policy', where the thresholds depend on both the estimation value and the state of the original process, and sampling and transmission need to be initiated when the instantaneous AoII exceeds the corresponding threshold, called the estimation- and state-aware transmission (ESAT) policy. Subsequently, we formulate the problem of finding the thresholds as a constrained semi-Markov decision process (CSMDP) and the Lagrangian approach. Additionally, we propose two lower complexity sub-optimum policies, namely the estimation-aware transmission (EAT) policy, and the single-threshold (ST) policy, for which it is possible to obtain these thresholds for CTMCs with relatively larger number of states. The underlying CSMDP formulation relies on the 'multi-regime phase-type' (MRPH) distribution which is a generalization of the well-known phase-type distribution, which allows us to obtain the distribution of time until absorption in a CTMC whose transition rates change with respect to time in a piece-wise manner. The effectiveness of the proposed ESAT, EAT and ST sampling and transmission policies are shown through numerical examples, along with comparisons with a baseline scheme that transmits packets according to a Poisson process in out-of-sync periods.

Figures (10)

• Figure 1: A remote estimation system with the source process $X(t)$ and the monitor process $\hat{X}(t)$ for which the source employs a certain transmission policy to transmit the status update packets via a delay channel, and also a preemption policy to preempt ongoing transmissions when the observed information becomes obsolete.
• Figure 2: Sample path of $X(t)$ (light grey lines), $\hat{X}(t)$ (red dotted lines), and $\text{AoII}(t)$ (thick red solid curve) for an example scenario when $N=2$ and $\text{AoII}(0)=0$. The arrows at $t=2$ and $t=9$ represent the reception epochs of status update packets at the monitor. Notice that $\text{AoII}(t)$ drops to zero at $t=7$ without a packet reception, as the process $X(t)$ returns on its own to the current estimate at the monitor $\hat{X}(t)$.
• Figure 3: A sample path for $X(t)$, $\hat{X}(t)$ and $\text{AoII}(t)$ for an example scenario. Green circles denote the synchronization points.
• Figure 4: MAoII depicted as a function of the single threshold $\tau$ when $N=20$, and for three values of the pair $(\sigma,\mu)$. Circles are used for simulation results whereas curves are used for analytical results. Optimum threshold values under the sampling constraints $b=0.4$ and $b=0.25$, respectively, are marked by black and turquoise diamonds.
• Figure 5: MAoII depicted as a function of $N$ for ST and PS policies when $b=0.25$ for different service rates. Lines correspond to analytical results, and circles are used for simulations.

Theorems & Definitions (6)

• Definition 1
• Theorem 1
• Definition 2
• Remark 1
• Definition 3
• Remark 2