Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Learning from Demonstration with Hierarchical Policy Abstractions Toward High-Performance and Courteous Autonomous Racing

Chanyoung Chung, Hyunki Seong, David Hyunchul Shim

TL;DR

An autonomous racing framework that learns complex racing behaviors from expert demonstrations using hierarchical policy abstractions that can overtake other vehicles by understanding nuanced interactions, effectively balancing performance and courtesy like professional drivers is proposed.

Abstract

Fully autonomous racing demands not only high-speed driving but also fair and courteous maneuvers. In this paper, we propose an autonomous racing framework that learns complex racing behaviors from expert demonstrations using hierarchical policy abstractions. At the trajectory level, our policy model predicts a dense distribution map indicating the likelihood of trajectories learned from offline demonstrations. The maximum likelihood trajectory is then passed to the control-level policy, which generates control inputs in a residual fashion, considering vehicle dynamics at the limits of performance. We evaluate our framework in a high-fidelity racing simulator and compare it against competing baselines in challenging multi-agent adversarial scenarios. Quantitative and qualitative results show that our trajectory planning policy significantly outperforms the baselines, and the residual control policy improves lap time and tracking accuracy. Moreover, challenging closed-loop experiments with ten opponents show that our framework can overtake other vehicles by understanding nuanced interactions, effectively balancing performance and courtesy like professional drivers.

Learning from Demonstration with Hierarchical Policy Abstractions Toward High-Performance and Courteous Autonomous Racing

TL;DR

An autonomous racing framework that learns complex racing behaviors from expert demonstrations using hierarchical policy abstractions that can overtake other vehicles by understanding nuanced interactions, effectively balancing performance and courtesy like professional drivers is proposed.

Abstract

Fully autonomous racing demands not only high-speed driving but also fair and courteous maneuvers. In this paper, we propose an autonomous racing framework that learns complex racing behaviors from expert demonstrations using hierarchical policy abstractions. At the trajectory level, our policy model predicts a dense distribution map indicating the likelihood of trajectories learned from offline demonstrations. The maximum likelihood trajectory is then passed to the control-level policy, which generates control inputs in a residual fashion, considering vehicle dynamics at the limits of performance. We evaluate our framework in a high-fidelity racing simulator and compare it against competing baselines in challenging multi-agent adversarial scenarios. Quantitative and qualitative results show that our trajectory planning policy significantly outperforms the baselines, and the residual control policy improves lap time and tracking accuracy. Moreover, challenging closed-loop experiments with ten opponents show that our framework can overtake other vehicles by understanding nuanced interactions, effectively balancing performance and courtesy like professional drivers.

Paper Structure

This paper contains 20 sections, 2 equations, 7 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Model-Predictive Control
  4. Game-Theoretic Planning
  5. Learning-based Approaches
  6. Problem Statement
  7. Methodology
  8. Trajectory Planning Policy
  9. Residual Control Policy
  10. Loss Function Design
  11. Implementation and Experiment
  12. Simulation Environment
  13. Dataset and Training
  14. Evaluation Results
  15. Baselines and Evaluation Metrics
  16. ...and 5 more sections

Figures (7)

  • Figure 1: Overtaking scenarios at the Indy Autonomous Challenge (left) and Formula 1 motorsport (right)
  • Figure 2: Overview of our autonomous racing framework.
  • Figure 3: Illustration of our trajectory-level policy model structure. Feature extractor takes inputs about the environment, neighboring opponents, and ego past trajectory information. Extracted contextual cues are fed into the parametric density estimator using NAF, and it finally outputs the distribution of future trajectory.
  • Figure 4:
  • Figure 6: Simulation environment for data collection and validation.
  • ...and 2 more figures