Chance-Aware Lane Change with High-Level Model Predictive Control Through Curriculum Reinforcement Learning

TL;DR

This work proposes a chance-aware lane-change strategy with high-level model predictive control (MPC) through curriculum reinforcement learning (CRL) through curriculum reinforcement learning (CRL), which generates high-quality lane-change maneuvers such that the vehicle merges into the traffic flow with a high success rate.

Abstract

Lane change in dense traffic typically requires the recognition of an appropriate opportunity for maneuvers, which remains a challenging problem in self-driving. In this work, we propose a chance-aware lane-change strategy with high-level model predictive control (MPC) through curriculum reinforcement learning (CRL). In our proposed framework, full-state references and regulatory factors concerning the relative importance of each cost term in the embodied MPC are generated by a neural policy. Furthermore, effective curricula are designed and integrated into an episodic reinforcement learning (RL) framework with policy transfer and enhancement, to improve the convergence speed and ensure a high-quality policy. The proposed framework is deployed and evaluated in numerical simulations of dense and dynamic traffic. It is noteworthy that, given a narrow chance, the proposed approach generates high-quality lane-change maneuvers such that the vehicle merges into the traffic flow with a high success rate of 96%. Finally, our framework is validated in the high-fidelity simulator under dense traffic, demonstrating satisfactory practicality and generalizability.

Figures (5)

• Figure 1: Overview of our proposed framework for chance-aware lane-change problems. The recognized dynamic chance for lane change is visually represented through the use of a red dashed rectangle.
• Figure 2: Reward curves of different methods. The training curves are smoothed by exponential moving average with a degree of 0.8, and the curriculum is switched at episodes 100 and 200.
• Figure 3: Key frames of a trail with our proposed approach for chance-aware lane change in numerical simulations. The vehicles on the middle and upper lane represent the traffic flow, the vehicle in red is the ego vehicle, the vehicles ahead of the ego vehicle on the lower lane are front vehicles, the dotted line in red and blue represent the future trajectory and the executed trajectory of the ego vehicle. The colorbars refer to the different longitudinal speed of predicted and executed trajectories of the ego vehicle.
• Figure 4: The speed and action profiles of a trail of chance-aware lane change.
• Figure 5: Key frames of the experimental validation of our method in the high-fidelity simulator. The top shows the third-person view attached to the ego vehicle. The bottom shows the bird-eye view, where the red rectangle is the ego vehicle while the green rectangles denote the surrounding vehicles.