From Freshness to Effectiveness: Goal-Oriented Sampling for Remote Decision Making

TL;DR

The paper develops AR-MDP, a framework that co-designs data sampling and remote decision making under random delays and a sampling-frequency constraint, treating AoI as a controllable and informative side-information metric. It introduces two low-complexity one-layer algorithms, OnePDSI and QuickBLP, to solve the unconstrained and rate-constrained problems, respectively, with rigorous convergence guarantees and exponential rates. The analysis reveals a fundamental sampling-frequency threshold: beyond this threshold, further sampling does not improve decision quality, highlighting a gap between freshness and actionable information. Simulation results demonstrate that goal-oriented sampling outperforms traditional AoI-based strategies across delay regimes, and clarify how decision-making benefits degrade as delays grow, driving the design of efficient communication-control policies for remote decision systems.

Abstract

Data freshness, measured by Age of Information (AoI), is highly relevant in networked applications such as Vehicle to Everything (V2X), smart health systems, and Industrial Internet of Things (IIoT). Yet, freshness alone does not equate to informativeness. In decision-critical settings, some stale data may prove more valuable than fresh updates. To explore this nuance, we move beyond AoI-centric policies and investigate how data staleness impacts decision-making under data-staleness-induced uncertainty. We pose a central question: What is the value of information, when freshness fades, and only its power to shape remote decisions remains? To capture this endured value, we propose AR-MDP, an Age-aware Remote Markov Decision Process framework, which co-designs optimal sampling and remote decision-making under a sampling frequency constraint and random delay. To efficiently solve this problem, we design a new two-stage hierarchical algorithm namely Quick Bellman-Linear-Program (QuickBLP), where the first stage involves solving the Dinkelbach root of a Bellman variant and the second stage involves solving a streamlined linear program (LP). For the tricky first stage, we propose a new One-layer Primal-Dinkelbach Synchronous Iteration (OnePDSI) method, which overcomes the re-convergence and non-expansive divergence present in existing per-sample multi-layer algorithms. Through rigorous convergence analysis of our proposed algorithms, we establish that the worst-case optimality gap in OnePDSI exhibits exponential decay with respect to iteration $K$ at a rate of $\mathcal{O}(\frac{1}{R^K})$. Through sensitivity analysis, we derive a threshold for the sampling frequency, beyond which additional sampling does not yield further gains in decision-making. Simulation results validate our analyses.

Figures (9)

• Figure 1: Comparisons among standard MDP, DDMDP, SDMDP, and AR-MDP.
• Figure 2: Evolution of the age $\Delta(t)$ over time.
• Figure 3: Bisection search to find the root of the implicit function $U(\lambda)$. The implicit function $U(\lambda)$ is approximated using a value iteration approach. The interval containing the root, denoted by $(\lambda_{\downarrow}, \lambda_{\uparrow})$, is halved at each outer-layer iteration, and this process eventually converges to the unique root $\rho^{\star}$.
• Figure 4: Algorithmic behavior of RVI versus $\tau$-RVI: Divergence mechanisms and comparative performance.
• Figure 5: Convergence comparison between FPBI li2024sampling and OnePDSI (Iteration \ref{['propsition2']}). FPBI becomes non-expansive and thus diverges. The proposed OnePDSI converges.

Theorems & Definitions (41)

• Definition 1
• Lemma 1
• proof
• Lemma 2
• proof
• Theorem 1
• proof
• Lemma 4
• Example 1
• Remark 1