Adaptive Autopilot: Constrained DRL for Diverse Driving Behaviors

TL;DR

The paper tackles replicating human-like driving in autonomous vehicles by learning in a constrained reinforcement-learning framework that respects safety while adapting to diverse driving styles. It introduces Adaptive Autopilot (AA), a three-part pipeline: (1) rule-based classification of real-world data into Aggressive, Normal, and Conservative driving, (2) per-style DNN regressors predicting human-like accelerations, and (3) a SAC-Lagrangian constrained DRL controller that minimizes deviations from the regressor while enforcing a headway-based safety constraint. Empirical results demonstrate accurate style classification, superior per-style acceleration prediction compared to IDM baselines, and safe, human-like driving behavior across styles with quantified RMSE improvements and headway adherence. The approach holds promise for safer, more trusted autopilots with reduced disengagement, and the authors outline future work on semi-supervised style categorization and richer driving scenarios.

Abstract

In pursuit of autonomous vehicles, achieving human-like driving behavior is vital. This study introduces adaptive autopilot (AA), a unique framework utilizing constrained-deep reinforcement learning (C-DRL). AA aims to safely emulate human driving to reduce the necessity for driver intervention. Focusing on the car-following scenario, the process involves (i) extracting data from the highD natural driving study and categorizing it into three driving styles using a rule-based classifier; (ii) employing deep neural network (DNN) regressors to predict human-like acceleration across styles; and (iii) using C-DRL, specifically the soft actor-critic Lagrangian technique, to learn human-like safe driving policies. Results indicate effectiveness in each step, with the rule-based classifier distinguishing driving styles, the regressor model accurately predicting acceleration, outperforming traditional car-following models, and C-DRL agents learning optimal policies for humanlike driving across styles.

Figures (9)

• Figure 1: A hierarchical rule-based classifier (partial representation for the case of headway less than or equal to 1 sec) that labels the driving data into three driving style categories: Aggressive, Normal, and Conservative.
• Figure 2: An overview of the proposed AA methodology.
• Figure 3: Human similarity (left) and comfort (right) reward trend.
• Figure 4: Analysis of driving styles classification using the highD dataset.
• Figure 5: Driver-wise regressor model predictions for the three driving styles.