Deep Reinforcement Learning for Local Path Following of an Autonomous Formula SAE Vehicle

TL;DR

This work investigates local path following for autonomous racing by comparing two RL paradigms: Soft Actor-Critic (SAC) and Adversarial Inverse Reinforcement Learning (AIRL). It introduces three reward functions tailored for autonomous racing, and evaluates both algorithms in a ROS-based Formula Student Driverless simulator with a realistic vehicle model. Results show SAC converges faster in simulation, while AIRL delivers more robust and reliable real-world inference and can operate at higher speeds, though both exhibit domain gaps relative to expert driver trajectories. The study highlights the sim-to-real challenges and identifies concrete avenues—reward shaping, transfer learning, and speed variation—that could enable scaling to a full-scale F:SAE vehicle with improved smoothness and safety in autonomous racing contexts.

Abstract

With the continued introduction of driverless events to Formula:Society of Automotive Engineers (F:SAE) competitions around the world, teams are investigating all aspects of the autonomous vehicle stack. This paper presents the use of Deep Reinforcement Learning (DRL) and Inverse Reinforcement Learning (IRL) to map locally-observed cone positions to a desired steering angle for race track following. Two state-of-the-art algorithms not previously tested in this context: soft actor critic (SAC) and adversarial inverse reinforcement learning (AIRL), are used to train models in a representative simulation. Three novel reward functions for use by RL algorithms in an autonomous racing context are also discussed. Tests performed in simulation and the real world suggest that both algorithms can successfully train models for local path following. Suggestions for future work are presented to allow these models to scale to a full F:SAE vehicle.

Figures (12)

• Figure 1: Auckland University F:SAE Team's 2019 electric vehicle.
• Figure 2: FS simulator visualisation - RVIZ.
• Figure 3: FSG 2019 full-scale trackdrive track in simulation.
• Figure 4: Reward function 2 visualisation.
• Figure 5: SAC vs AIRL simulation inference performance - 120 trials per variant.