Optimizing Information Freshness in IoT Systems with Update Rate Constraints: A Token-Based Approach

TL;DR

This work tackles freshness optimization in IoT status-update systems under update-rate constraints by transforming CMDPs into unconstrained MDPs via a token-based mechanism inspired by token buckets. It derives a token-based unconstrained MDP for AoII under a single rate constraint and extends it to AoI under two rate constraints, augmented by an iterative triangle bisection algorithm to solve the two-constraint CMDP. Theoretical results show the optimal token-based policy has a threshold structure, and numerical results demonstrate that token-based policies converge to CMDP-optimal performance as the token budget increases, often outperforming baselines. The approach offers a practical, scalable framework for managing information freshness in resource-limited IoT networks with multiple rate constraints, enabling standard MDP solvers to be applied directly.

Abstract

In Internet of Things (IoT) status update systems, where information is sampled and subsequently transmitted from a source to a destination node, the imperative necessity lies in maintaining the timeliness of information and updating the system with optimal frequency. Optimizing information freshness in resource-limited status update systems often involves Constrained Markov Decision Process (CMDP) problems with update rate constraints. Solving CMDP problems, especially with multiple constraints, is a challenging task. To address this, we present a token-based approach that transforms CMDP into an unconstrained MDP, simplifying the solution process. We apply this approach to systems with one and two update rate constraints for optimizing Age of Incorrect Information (AoII) and Age of Information (AoI) metrics, respectively, and explore the analytical and numerical aspects. Additionally, we introduce an iterative triangle bisection method for solving the CMDP problems with two constraints, comparing its results with the token-based MDP approach. Our findings show that the token-based approach yields superior performance over baseline policies, converging to the optimal policy as the maximum number of tokens increases.

Figures (6)

• Figure 1: A general system model for optimizing information freshness under update rate constraints.
• Figure 2: Optimal average AoII for two policies vs. $\alpha$ in the single rate constrained problem.
• Figure 3: Optimal average AoII for two policies vs. $p_R$ in the single rate constrained problem.
• Figure 4: Average AoI for different policies vs. $q$ ($\alpha_{\text{max}}=0.5$) in the two rate constrained problem.
• Figure 5: Average AoI for different policies vs. $\alpha_{\text{max}}$ ($q=0.2$) in the two rate constrained problem.

Theorems & Definitions (15)

• Definition 1
• Proposition 1
• proof
• Proposition 2
• proof
• Definition 2
• Theorem 1
• proof
• Proposition 3
• proof