Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing AoI at Query in Multiuser Wireless Uplink Networks: A Whittle Index Approach

Jingwei Liu, He Chen

TL;DR

This paper uses the age of information at query (QAoI) to characterize the performance of information freshness in a pull-based wireless network and devise an efficient algorithm to calculate Whittle indices for the formulated sub-MDPs.

Abstract

In this paper, we explore how to schedule multiple users to optimize information freshness in a pull-based wireless network, where the status updates from users are requested by randomly arriving queries at the destination. We use the age of information at query (QAoI) to characterize the performance of information freshness. Such a decision-making problem is naturally modeled as a Markov decision process (MDP), which, however, is prohibitively high to be solved optimally by the standard method due to the curse of dimensionality. To address this issue, we employ Whittle index approach, which allows us to decouple the original MDP into multiple sub-MDPs by relaxing the scheduling constraints. However, the binary Markovian query arrival process results in a bi-dimensional state and complex state transitions within each sub-MDP, making it challenging to verify Whittle indexability using conventional methods. After a thorough analysis of the sub-MDP's structure, we show that it is unichain and its optimal policy follows a threshold-type structure. This facilitates the verification of Whittle indexability of the sub-MDP by employing an easy-to-verify condition. Subsequently, the steady-state probability distributions of the sub-MDP under different threshold-type policies are analyzed, constituting the analytical expressions of different Whittle indices in terms of the expected average QAoI and scheduling time of the sub-MDP. Building on these, we devise an efficient algorithm to calculate Whittle indices for the formulated sub-MDPs. The simulation results validate our analyses and show the proposed Whittle index policy outperforms baseline policies and achieves near-optimal performance.

Optimizing AoI at Query in Multiuser Wireless Uplink Networks: A Whittle Index Approach

TL;DR

This paper uses the age of information at query (QAoI) to characterize the performance of information freshness in a pull-based wireless network and devise an efficient algorithm to calculate Whittle indices for the formulated sub-MDPs.

Abstract

In this paper, we explore how to schedule multiple users to optimize information freshness in a pull-based wireless network, where the status updates from users are requested by randomly arriving queries at the destination. We use the age of information at query (QAoI) to characterize the performance of information freshness. Such a decision-making problem is naturally modeled as a Markov decision process (MDP), which, however, is prohibitively high to be solved optimally by the standard method due to the curse of dimensionality. To address this issue, we employ Whittle index approach, which allows us to decouple the original MDP into multiple sub-MDPs by relaxing the scheduling constraints. However, the binary Markovian query arrival process results in a bi-dimensional state and complex state transitions within each sub-MDP, making it challenging to verify Whittle indexability using conventional methods. After a thorough analysis of the sub-MDP's structure, we show that it is unichain and its optimal policy follows a threshold-type structure. This facilitates the verification of Whittle indexability of the sub-MDP by employing an easy-to-verify condition. Subsequently, the steady-state probability distributions of the sub-MDP under different threshold-type policies are analyzed, constituting the analytical expressions of different Whittle indices in terms of the expected average QAoI and scheduling time of the sub-MDP. Building on these, we devise an efficient algorithm to calculate Whittle indices for the formulated sub-MDPs. The simulation results validate our analyses and show the proposed Whittle index policy outperforms baseline policies and achieves near-optimal performance.

Paper Structure

This paper contains 24 sections, 8 theorems, 109 equations, 8 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. System Model and Problem Formulation
  3. Network Model
  4. Information Freshness Metric and Problem Formulation
  5. MDP Formulation and Relaxation
  6. MDP Components
  7. MDP Relaxation for the Whittle Index approach
  8. Whittle Indexability
  9. Algorithms to calculate Whittle Index
  10. Algorithm for Error-Prone Networks
  11. Simplified Algorithm for Error-Free Networks
  12. Numerical Results
  13. Verification of the Analysis
  14. Comparisons with Baseline Policies
  15. Conclusion
  16. ...and 9 more sections

Key Result

Proposition 1

The formulated sub-MDP is an unichain MDP, which has the following properties. Given a stationary deterministic policy $\pi$ for a sub-MDP, there exist a scalar $R^{\pi}$ and a function $h$ that satisfy the following Bellman’s equation where $R^{\pi}$ is the expected average reward under policy $\pi$ and satisfies and $h^{\pi}(s)$ is the relative cost function, presented as for a reference stat

Figures (8)

  • Figure 1: The wireless multiuser uplink network with Markovian query arrival processes at the monitors.
  • Figure 2: An example of a sub-MDP applying a thresh-type policy $\pi(H_0;H_1)$ with $H_1<H_0<D_m$, $p<1$.
  • Figure 3: An example of a sub-MDP applying a thresh-type policy $\pi(H_0;H_1)$ with $H_1<H_0\le D_m$, $p=1$.
  • Figure 4: The optimal policy of a sub-MDP versus the scheduling cost $C$, where $\lambda=0.4$, $\gamma=0.3$, $p=0.7$, and $D_m=50$.
  • Figure 5: The optimal policy of a sub-MDP versus the scheduling cost $C$, where $\lambda=0.4$, $\gamma=0.6$, $p=0.7$, and $D_m=50$.
  • ...and 3 more figures

Theorems & Definitions (18)

  • Proposition 1
  • proof
  • Remark 1
  • Definition 1
  • Definition 2
  • Lemma 1
  • proof
  • Theorem 1
  • proof
  • Corollary 1
  • ...and 8 more