Optimizing Version AoI in Energy-Harvesting IoT: Model-Based and Learning-Based Approaches

TL;DR

The paper addresses optimizing Version Age of Information (VAoI) in an energy-harvesting IoT status-update system under three information regimes: fully known, known with unknown parameters, and completely unknown. It leverages Markov Decision Process (MDP) and Reinforcement Learning (RL) frameworks to develop model-based, estimation-based, and model-free solutions, with Relative Value Iteration (RVIA) used for the fully known case and Q-learning for unknown models. Key findings show a threshold-based optimal policy in the fully known setting, faster convergence for estimation-based methods when parameters can be estimated, and the viability of model-free RL in completely unknown environments at the expense of longer learning times. The results provide actionable guidance for designing efficient, semantics-aware transmission policies in energy-constrained IoT deployments, and point to future work on integrating threshold structures with RL and exploring Deep RL for scalability.

Abstract

Efficient data transmission in resource-constrained Internet of Things (IoT) systems requires semantics-aware management that maximizes the delivery of timely and informative data. This paper investigates the optimization of the semantic metric Version Age of Information (VAoI) in a status update system comprising an energy-harvesting (EH) sensor and a destination monitoring node. We consider three levels of knowledge about the system model -- fully known, partially known, and unknown -- and propose corresponding optimization strategies: model-based, estimation-based, and model-free methods. By employing Markov Decision Process (MDP) and Reinforcement Learning (RL) frameworks, we analyze performance trade-offs under varying degrees of model information. Our findings provide guidance for designing efficient and adaptive semantics-aware policies in both known and unknown IoT environments.

Figures (5)

• Figure 1: The status update system model.
• Figure 2: Optimal Actions for each state $(\Delta,b)$.
• Figure 3: Optimal average VAoI vs. $\beta$.
• Figure 4: Average VAoI for different approaches with $\beta=0.2$.
• Figure 5: Average VAoI for different approaches with $\beta=0.1$.