Developing Driving Strategies Efficiently: A Skill-Based Hierarchical Reinforcement Learning Approach

TL;DR

This letter proposes “skill-based” hierarchical driving strategies, where motion primitives, i.e., skills, are designed and used as high-level actions, which reduces the training time for applications that require multiple models with varying behavior.

Abstract

Driving in dense traffic with human and autonomous drivers is a challenging task that requires high-level planning and reasoning. Human drivers can achieve this task comfortably, and there has been many efforts to model human driver strategies. These strategies can be used as inspirations for developing autonomous driving algorithms or to create high-fidelity simulators. Reinforcement learning is a common tool to model driver policies, but conventional training of these models can be computationally expensive and time-consuming. To address this issue, in this paper, we propose ``skill-based" hierarchical driving strategies, where motion primitives, i.e. skills, are designed and used as high-level actions. This reduces the training time for applications that require multiple models with varying behavior. Simulation results in a merging scenario demonstrate that the proposed approach yields driver models that achieve higher performance with less training compared to baseline reinforcement learning methods.

Figures (5)

• Figure 1: NGSIM I-80 Study Area, Area of Interest Enclosed in Red Lines
• Figure 2: Highway Merging Environment
• Figure 3: Example skill 1.
• Figure 4: Example skill 2.
• Figure 5: HRL and Baseline Model's Reward and Finish Rates vs. Training Time