Non-orthogonal Age-Optimal Information Dissemination in Vehicular Networks: A Meta Multi-Objective Reinforcement Learning Approach

TL;DR

This article considers minimizing the age-of-information (AoI) and transmit power consumption in a vehicular network, where a roadside unit (RSU) provides timely updates about a set of physical processes to vehicles, and presents a two-stage meta-multi-objective reinforcement learning solution to estimate the Pareto front.

Abstract

This paper considers minimizing the age-of-information (AoI) and transmit power consumption in a vehicular network, where a roadside unit (RSU) provides timely updates about a set of physical processes to vehicles. We consider non-orthogonal multi-modal information dissemination, which is based on superposed message transmission from RSU and successive interference cancellation (SIC) at vehicles. The formulated problem is a multi-objective mixed-integer nonlinear programming problem; thus, a Pareto-optimal front is very challenging to obtain. First, we leverage the weighted-sum approach to decompose the multi-objective problem into a set of multiple single-objective sub-problems corresponding to each predefined objective preference weight. Then, we develop a hybrid deep Q-network (DQN)-deep deterministic policy gradient (DDPG) model to solve each optimization sub-problem respective to predefined objective-preference weight. The DQN optimizes the decoding order, while the DDPG solves the continuous power allocation. The model needs to be retrained for each sub-problem. We then present a two-stage meta-multi-objective reinforcement learning solution to estimate the Pareto front with a few fine-tuning update steps without retraining the model for each sub-problem. Simulation results illustrate the efficacy of the proposed solutions compared to the existing benchmarks and that the meta-multi-objective reinforcement learning model estimates a high-quality Pareto frontier with reduced training time.

Figures (16)

• Figure 1: System model with $V=4$ vehicles, $F=4$ processes, information demand $\mathbf{R}$ in \ref{['demR']}, and a decoding order decision $\bm{\pi}^{(t)} = [1, 2, 3, 4]$.
• Figure 2: Hybrid DQN-DDPG model for age-optimum information dissemination.
• Figure 3: The learning curves for the hybrid DQN-DDPG DRL model versus the number of training episodes.
• Figure 4: Objective function in \ref{['Obj']} versus the relative weight $\zeta$ with $V=10$ vehicles, $F=4$ processes and $\abs{\mathcal{R}_i}=2$.
• Figure 5: Average AoI and power expenditure versus the relative weight $\zeta$ with $V=10$ vehicles, processes $F=4$ processes and $\abs{\mathcal{R}_i}=2$.