Beyond Martingale Estimators: Structured Estimators for Maximizing Information Freshness in Query-Based Update Systems

TL;DR

The paper addresses MBF-based information freshness in query-based remote estimation of CTMC sources, challenging the conventional martingale estimator by introducing structured estimators that interpolate toward MAP. It introduces the $p$-MAP estimator and other two-stage and multi-stage schemes, derives closed-form MBF expressions (notably for time-reversible CTMCs), and develops an SMDP framework for optimal state-dependent sampling, plus an approach for rate allocation across multiple CTMCs. Numerical results show significant MBF gains over ME, with additional improvements from adaptive sampling policies. These contributions provide a practical, analytically tractable toolkit for sustaining freshness in pull-based monitoring of heterogeneous CTMC sources, with potential extensions to broader freshness metrics such as AoI.

Abstract

This paper investigates information freshness in a remote estimation system in which the remote information source is a continuous-time Markov chain (CTMC). For such systems, estimators have been mainly restricted to the class of martingale estimators in which the remote estimate at any time is equal to the value of the most recently received update. This is mainly due to the simplicity and ease of analysis of martingale estimators, which however are far from optimal, especially in query-based (i.e., pull-based) update systems. In such systems, maximum a-posteriori probability (MAP) estimators are optimal. However, MAP estimators can be challenging to analyze in continuous-time settings. In this paper, we introduce a new class of estimators, called structured estimators, which can seamlessly shift from a martingale estimator to a MAP estimator, enabling them to retain useful characteristics of the MAP estimate, while still being analytically tractable. Particularly, we introduce a new estimator termed as the $p$-MAP estimator which is a piecewise-constant approximation of the MAP estimator with finitely many discontinuities, bringing us closer to a full characterization of MAP estimators when modeling information freshness. In fact, we show that for time-reversible CTMCs, the MAP estimator reduces to a $p$-MAP estimator. Using the binary freshness (BF) process for the characterization of information freshness, we derive the freshness expressions and provide optimal state-dependent sampling policies (i.e., querying policies) for maximizing the mean BF (MBF) for pull-based remote estimation of a single CTMC information source, when structured estimators are used. Moreover, we provide optimal query rate allocation policies when a monitor pulls information from multiple heterogeneous CTMCs with a constraint on the overall query rate.

Figures (15)

• Figure 1: Query-based sampling of CTMCs. In this example, the remote CTMC represents the state of a compute unit, which takes three values $\{\text{Sleep}, \text{Idle}, \text{Process}\}$. The true state of the CTMC at time $t$ is $X(t)$. The remote monitor samples (sends queries) to the CTMC, and gets instantaneous responses (readings of the state). Based on the responses, the monitor keeps an estimate of the state of the remote CTMC at time $t$ as $\hat{X}(t)$.
• Figure 2: Examples of time-reversible CTMC structures.
• Figure 3: (a) A CTMC with a unique stationary maximum. Here, the stationary distribution is $\bm\pi=\{0.18,0.27,0.55\}$, and hence, $i^*=\mathop{\mathrm{arg\,max}}\limits_{i\in \mathcal{S}}\pi_i=3$. (b) Note that, for all $t$ after $\tau^*$, we have $\mathop{\mathrm{arg\,max}}\limits_i P_{ji}(t)=\mathop{\mathrm{arg\,max}}\limits_{i \in \mathcal{S}} \bm{e}_j^T P(t)=i^*=3$. The figure shows the case for $j=1$, i.e., the CTMC starts from state 1 at time zero.
• Figure 4: (a) An infinitely oscillating CTMC without a unique stationary maximum. Here, the stationary distribution is $\bm\pi=\{\frac{1}{3},\frac{1}{3},\frac{1}{3}\}$, with no unique $i^*=\mathop{\mathrm{arg\,max}}\limits_{i\in \mathcal{S}}\pi_i$. (b) There is no time $\tau^*$ after which $\mathop{\mathrm{arg\,max}}\limits_i P_{ji}(t)$ is constant, in fact, it is infinitely oscillating.
• Figure 5: Evolution of the estimate $\hat{X}(t)$ at the remote monitor before receiving an update given $\hat{X}(0)=1$. Here, $\tau^*_{1,k}$ denotes the $k$th transition point of the MAP estimator.

Theorems & Definitions (27)

• Lemma 1
• Definition 1
• Remark 1
• Lemma 2
• Lemma 3
• Remark 2
• Remark 3
• Theorem 1
• Remark 4
• Theorem 2