Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Towards an Autonomous Test Driver: High-Performance Driver Modeling via Reinforcement Learning

John Subosits, Jenna Lee, Shawn Manuel, Paul Tylkin, Avinash Balachandran

TL;DR

This work addresses rapid evaluation of racing-car configurations by learning an autonomous test driver via deep reinforcement learning. It exposes a multi-task DSAC-based policy that conditions on vehicle setup to drive near human-level performance across multiple configurations, with optional imitation objectives to bias behavior toward human demonstrations. The results show the agent can surpass a professional driver in several setups, generalize to unseen configurations, and do so with a single shared policy, enabling efficient setup sensitivity analysis in high-fidelity simulators. Overall, the approach advances simulation-driven vehicle development by enabling robust, driver-aware evaluation of setup changes with potential for driver-specific optimization.

Abstract

Success in racing requires a unique combination of vehicle setup, understanding of the racetrack, and human expertise. Since building and testing many different vehicle configurations in the real world is prohibitively expensive, high-fidelity simulation is a critical part of racecar development. However, testing different vehicle configurations still requires expert human input in order to evaluate their performance on different racetracks. In this work, we present the first steps towards an autonomous test driver, trained using deep reinforcement learning, capable of evaluating changes in vehicle setup on racing performance while driving at the level of the best human drivers. In addition, the autonomous driver model can be tuned to exhibit more human-like behavioral patterns by incorporating imitation learning into the RL training process. This extension permits the possibility of driver-specific vehicle setup optimization.

Towards an Autonomous Test Driver: High-Performance Driver Modeling via Reinforcement Learning

TL;DR

This work addresses rapid evaluation of racing-car configurations by learning an autonomous test driver via deep reinforcement learning. It exposes a multi-task DSAC-based policy that conditions on vehicle setup to drive near human-level performance across multiple configurations, with optional imitation objectives to bias behavior toward human demonstrations. The results show the agent can surpass a professional driver in several setups, generalize to unseen configurations, and do so with a single shared policy, enabling efficient setup sensitivity analysis in high-fidelity simulators. Overall, the approach advances simulation-driven vehicle development by enabling robust, driver-aware evaluation of setup changes with potential for driver-specific optimization.

Abstract

Success in racing requires a unique combination of vehicle setup, understanding of the racetrack, and human expertise. Since building and testing many different vehicle configurations in the real world is prohibitively expensive, high-fidelity simulation is a critical part of racecar development. However, testing different vehicle configurations still requires expert human input in order to evaluate their performance on different racetracks. In this work, we present the first steps towards an autonomous test driver, trained using deep reinforcement learning, capable of evaluating changes in vehicle setup on racing performance while driving at the level of the best human drivers. In addition, the autonomous driver model can be tuned to exhibit more human-like behavioral patterns by incorporating imitation learning into the RL training process. This extension permits the possibility of driver-specific vehicle setup optimization.

Paper Structure

This paper contains 19 sections, 7 equations, 11 figures, 6 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Formulation and Approach
  4. State and Action Spaces
  5. Reward Function Design
  6. Policy Learning
  7. Experimental Results
  8. Analysis of Agent's Driving Across Setups
  9. Comparison to Human Driving
  10. Imitation of Human Demonstrations
  11. Encouraging Open Loop Similarity
  12. Encouraging Closed Loop Similarity
  13. Results
  14. Conclusion
  15. Network Architecture
  16. ...and 4 more sections

Figures (11)

  • Figure 1: Illustration of the state space and its components.
  • Figure 2: Comparison of agent's speed in each of the four tests showing primarily subtle differences but a qualitatively correct trend.
  • Figure 3: Comparison of agent's steering input in each of the four tests showing adjustments for oversteer and understeer.
  • Figure 4: Comparison of agent's brake point for Turn 1 with changing friction conditions.
  • Figure 5: Gain of time relative to the human driver for the fastest setup where the difference is smallest.
  • ...and 6 more figures