Joint Channel Selection using FedDRL in V2X

TL;DR

This paper proposes an approach based on Federated Deep Reinforcement Learning (FedDRL), which enables each vehicle to benefit from other vehicles’ experiences in the problem of joint channel selection, and applies the federated Proximal Policy Optimization (FedPPO) algorithm to this task.

Abstract

Vehicle-to-everything (V2X) communication technology is revolutionizing transportation by enabling interactions between vehicles, devices, and infrastructures. This connectivity enhances road safety, transportation efficiency, and driver assistance systems. V2X benefits from Machine Learning, enabling real-time data analysis, better decision-making, and improved traffic predictions, making transportation safer and more efficient. In this paper, we study the problem of joint channel selection, where vehicles with different technologies choose one or more Access Points (APs) to transmit messages in a network. In this problem, vehicles must learn a strategy for channel selection, based on observations that incorporate vehicles' information (position and speed), network and communication data (Signal-to-Interference-plus-Noise Ratio from past communications), and environmental data (road type). We propose an approach based on Federated Deep Reinforcement Learning (FedDRL), which enables each vehicle to benefit from other vehicles' experiences. Specifically, we apply the federated Proximal Policy Optimization (FedPPO) algorithm to this task. We show that this method improves communication reliability while minimizing transmission costs and channel switches. The efficiency of the proposed solution is assessed via realistic simulations, highlighting the potential of FedDRL to advance V2X technology.

Figures (3)

• Figure 1: Examples of the three traffic environments: countryside (left), highway (middle), and urban (right). The platoon involves a Follower (blue) and a Leader (orange), with background traffic (green). Countryside routes include multiple hairpin bends; highways are straightforward lines with good visibility; and urban routes is a grid layout with buildings that restrict the field of sight.
• Figure 2: Cumulative rewards as a function of the number of episodes in non-federated and federated settings across three different traffic scenarios: highway, urban, and countryside.
• Figure 3: Decisions of the follower in the countryside environment based on the relative distance between Leader and the Follower. From upper left: (i) policy learned on the highway without background traffic, (ii) federated policy without background traffic, (iii) baseline policy learned on the highway with background traffic, (iv) federated policy with background traffic.