Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey of Freshness-Aware Wireless Networking with Reinforcement Learning

Alimu Alibotaiken, Suyang Wang, Oluwaseun T. Ajayi, Yu Cheng

TL;DR

This paper consolidates RL-driven freshness optimization by reframing AoI into native, function-based, and application-aware metrics. It proposes a policy-centric taxonomy—update-control, medium-access, and risk-sensitive RL—complemented by MARL insights to address centralized, decentralized, and CTDE architectures across diverse wireless scenarios. The survey synthesizes RL and DRL methods applied to sampling, scheduling, trajectory planning, MAC access, and multi-agent coordination, highlighting methodological advances and open challenges such as delayed feedback and cross-layer design. By linking fidelity of information with task performance and reliability, it provides a unified foundation for learning-based freshness control in next-generation networks. The work aims to guide future research toward robust, scalable, and application-driven RL frameworks for AoI in B5G/6G environments.

Abstract

The age of information (AoI) has become a central measure of data freshness in modern wireless systems, yet existing surveys either focus on classical AoI formulations or provide broad discussions of reinforcement learning (RL) in wireless networks without addressing freshness as a unified learning problem. Motivated by this gap, this survey examines RL specifically through the lens of AoI and generalized freshness optimization. We organize AoI and its variants into native, function-based, and application-oriented families, providing a clearer view of how freshness should be modeled in B5G and 6G systems. Building on this foundation, we introduce a policy-centric taxonomy that reflects the decisions most relevant to freshness, consisting of update-control RL, medium-access RL, risk-sensitive RL, and multi-agent RL. This structure provides a coherent framework for understanding how learning can support sampling, scheduling, trajectory planning, medium access, and distributed coordination. We further synthesize recent progress in RL-driven freshness control and highlight open challenges related to delayed decision processes, stochastic variability, and cross-layer design. The goal is to establish a unified foundation for learning-based freshness optimization in next-generation wireless networks.

A Survey of Freshness-Aware Wireless Networking with Reinforcement Learning

TL;DR

This paper consolidates RL-driven freshness optimization by reframing AoI into native, function-based, and application-aware metrics. It proposes a policy-centric taxonomy—update-control, medium-access, and risk-sensitive RL—complemented by MARL insights to address centralized, decentralized, and CTDE architectures across diverse wireless scenarios. The survey synthesizes RL and DRL methods applied to sampling, scheduling, trajectory planning, MAC access, and multi-agent coordination, highlighting methodological advances and open challenges such as delayed feedback and cross-layer design. By linking fidelity of information with task performance and reliability, it provides a unified foundation for learning-based freshness control in next-generation networks. The work aims to guide future research toward robust, scalable, and application-driven RL frameworks for AoI in B5G/6G environments.

Abstract

The age of information (AoI) has become a central measure of data freshness in modern wireless systems, yet existing surveys either focus on classical AoI formulations or provide broad discussions of reinforcement learning (RL) in wireless networks without addressing freshness as a unified learning problem. Motivated by this gap, this survey examines RL specifically through the lens of AoI and generalized freshness optimization. We organize AoI and its variants into native, function-based, and application-oriented families, providing a clearer view of how freshness should be modeled in B5G and 6G systems. Building on this foundation, we introduce a policy-centric taxonomy that reflects the decisions most relevant to freshness, consisting of update-control RL, medium-access RL, risk-sensitive RL, and multi-agent RL. This structure provides a coherent framework for understanding how learning can support sampling, scheduling, trajectory planning, medium access, and distributed coordination. We further synthesize recent progress in RL-driven freshness control and highlight open challenges related to delayed decision processes, stochastic variability, and cross-layer design. The goal is to establish a unified foundation for learning-based freshness optimization in next-generation wireless networks.
Paper Structure (37 sections, 9 equations, 6 figures, 8 tables)

This paper contains 37 sections, 9 equations, 6 figures, 8 tables.

Table of Contents

  1. Introduction
  2. Background and Preliminaries
  3. AoI fundamentals
  4. Supervised Learning for AoI Optimization
  5. Core Concepts of Reinforcement Learning
  6. Agent--Environment Interaction and Markov Decision Processes
  7. Policy and Value Functions
  8. Bellman Equations and Optimality
  9. Dynamic Programming
  10. From Model-Based to Model-Free Methods
  11. Tabular Solution Methods
  12. Policy Gradient Methods
  13. From RL to DRL
  14. Core concepts of Multi-Agent Reinforcement Learning
  15. Advantage of RL over Traditional Optimization
  16. ...and 22 more sections

Figures (6)

  • Figure 1: Overall structure of this survey.
  • Figure 2: Sawtooth age waveform.
  • Figure 3: General wireless systems.
  • Figure 4: The architecture of MAC components from article1.
  • Figure 5: Comparison between centralized, fully decentralized, and CTDE architectures.
  • ...and 1 more figures