Age-of-information minimization via opportunistic sampling by an energy harvesting source

TL;DR

This work studies minimizing the time-average AoI in energy-harvesting systems where the transmitter can probe the channel before deciding to sample and send updates. Formulated as an infinite-horizon MDP with a two-stage action model (channel probing and sampling), the analysis yields threshold-based structure for i.i.d. channels and extends to Markovian channel and EH processes. When dynamics are unknown, the authors develop two-stage Q-learning algorithms suitable for both i.i.d. and Markovian settings, and demonstrate convergence and policy efficacy via numerical results. The findings highlight the value of channel probing in reducing AoI under practical energy-energy trade-offs and provide actionable RL methods for real-world EH remote sensing networks.

Abstract

Herein, minimization of time-averaged age-of-information (AoI) in an energy harvesting (EH) source setting is considered. The EH source opportunistically samples one or multiple processes over discrete time instants and sends the status updates to a sink node over a wireless fading channel. Each time, the EH node decides whether to probe the link quality and then decides whether to sample a process and communicate based on the channel probe outcome. The trade-off is between the freshness of information available at the sink node and the available energy at the source node. We use infinite horizon Markov decision process (MDP) to formulate the AoI minimization problem for two scenarios where energy arrival and channel fading processes are: (i) independent and identically distributed (i.i.d.), (ii) Markovian. In i.i.d. setting, after channel probing, the optimal source sampling policy is shown to be a threshold policy. Also, for unknown channel state and EH characteristics, a variant of the Q-learning algorithm is proposed for the two-stage action model, that seeks to learn the optimal policy. For Markovian system, the problem is again formulated as an MDP, and a learning algorithm is provided for unknown dynamics. Finally, numerical results demonstrate the policy structures and performance trade-offs.

Figures (8)

• Figure 1: Pictorial representation of a remote sensing system where an EH source samples one of $N$ number of processes at a time and sends the observation packet to a sink node.
• Figure 2: Single process ($N=1$), I.I.D. model, known dynamics: (a) Variation of $T_{th}(E)$ with $E, \lambda$ and (b) Variation of $p_{th}(E,T)$ with $E, T, \lambda$.
• Figure 3: $N=3$, I.I.D. model, known dynamics: (a) Variation of $T_{th}(E,T_2, T_3)$ with $E, T_2,T_3, \lambda$ and (b) Variation of $p_{th}(E,T_1, T_2, T_3)$ with $E,T_1,T_2, T_3, \lambda$.
• Figure 4: $N=1$, Markov model: (a) Variation of $T_{th}(E,\tau, C_{prev},H)$ with $E$, $\tau$, $C_{prev}=C_{1}$ and $H$ and (b) Variation of $T_{th}(E,\tau, C_{prev},H)$ with $E$, $\tau$, $C_{prev}=C_{2}$ and $H$.
• Figure 5: $N=1$, Markov model: Variation of $p_{th}(E,T,H)$ with $E$, $T$, and $H$.

Theorems & Definitions (16)

• Proposition 1
• proof
• Lemma 1
• proof
• Conjecture 1
• Remark 1
• Theorem 1
• proof
• Lemma 2
• proof