Meta Learning Based Adaptive Cooperative Perception in Nonstationary Vehicular Networks

TL;DR

A meta RL solution is proposed, which trains a meta RL model that captures the general features among LVNs, thus facilitating fast model adaptation for each LVN with the meta RL model as an initial point, and results show the superiority of meta RL in terms of the convergence speed without reward degradation.

Abstract

To accommodate high network dynamics in real-time cooperative perception (CP), reinforcement learning (RL) based adaptive CP schemes have been proposed, to allow adaptive switchings between CP and stand-alone perception modes among connected and autonomous vehicles. The traditional offline-training online-execution RL framework suffers from performance degradation under nonstationary network conditions. To achieve fast and efficient model adaptation, we formulate a set of Markov decision processes for adaptive CP decisions in each stationary local vehicular network (LVN). A meta RL solution is proposed, which trains a meta RL model that captures the general features among LVNs, thus facilitating fast model adaptation for each LVN with the meta RL model as an initial point. Simulation results show the superiority of meta RL in terms of the convergence speed without reward degradation. The impact of the customization level of meta models on the model adaptation performance has also been evaluated.

Figures (4)

• Figure 1: Vehicular networks with spatio-temporal nonstationary network dynamics.
• Figure 2: Performance comparison between training from scratch and meta adaptation under different workloads.
• Figure 3: Performance comparison between training from scratch, transfer learning, and meta learning for one task.
• Figure 4: Model adaptation performance with meta models of different customization levels.