Accelerating Real-World Overtaking in F1TENTH Racing Employing Reinforcement Learning Methods

TL;DR

This paper tackles the challenge of autonomous wheel-to-wheel racing and overtaking by training an end-to-end TD3 reinforcement learning agent in a high-fidelity F1Tenth simulator against opponents. The agent learns to overtake while racing, and is subsequently deployed on a real F1Tenth vehicle, achieving an 87% overtaking rate in real-world overtaking scenarios, outperforming a raceline-trained baseline at 56%. Simulation results show TD3-Overtake reaching an 88% success rate versus 55% for TD3-Race, indicating the benefits of opponent-aware training. The work demonstrates a strong sim-to-real transfer with minimal changes, reducing the gap between simulated and real driving, and suggests future directions toward higher speeds and broader comparative evaluations.

Abstract

While autonomous racing performance in Time-Trial scenarios has seen significant progress and development, autonomous wheel-to-wheel racing and overtaking are still severely limited. These limitations are particularly apparent in real-life driving scenarios where state-of-the-art algorithms struggle to safely or reliably complete overtaking manoeuvres. This is important, as reliable navigation around other vehicles is vital for safe autonomous wheel-to-wheel racing. The F1Tenth Competition provides a useful opportunity for developing wheel-to-wheel racing algorithms on a standardised physical platform. The competition format makes it possible to evaluate overtaking and wheel-to-wheel racing algorithms against the state-of-the-art. This research presents a novel racing and overtaking agent capable of learning to reliably navigate a track and overtake opponents in both simulation and reality. The agent was deployed on an F1Tenth vehicle and competed against opponents running varying competitive algorithms in the real world. The results demonstrate that the agent's training against opponents enables deliberate overtaking behaviours with an overtaking rate of 87% compared 56% for an agent trained just to race.

Figures (10)

• Figure 1: F1Tenth vehicle beginning an overtake manoeuvre on an opponent, moving wide to go around the competition. Full video of overtaking manoeuvres is available at https://youtu.be/6vRRWeZTG-k
• Figure 2: Training tracks in simulation. The models were designed in CAD using a spline function, which was then extruded to widths ranging from 1.5m to 3.5m. The figure shows these models as they are rendered in the Gazebo environment.
• Figure 3: TD3-Overtake training reward over 140,000 training steps in the simulation.
• Figure 4: Simulation testing track with track width of 2.5m.
• Figure 5: Graph of simulation overtaking results. Each ego vehicle algorithm had 100 attempts to overtake each competitor algorithm.