Reference-Free Formula Drift with Reinforcement Learning: From Driving Data to Tire Energy-Inspired, Real-World Policies

TL;DR

This work designs a reinforcement learning agent that builds on the concept of tire energy absorption to autonomously drift through changing and complex waypoint configurations while safely staying within track bounds and achieves zero-shot deployment on the car by training the agent in a simulation environment built on top of a neural stochastic differential equation vehicle model learned from pre-collected driving data.

Abstract

The skill to drift a car--i.e., operate in a state of controlled oversteer like professional drivers--could give future autonomous cars maximum flexibility when they need to retain control in adverse conditions or avoid collisions. We investigate real-time drifting strategies that put the car where needed while bypassing expensive trajectory optimization. To this end, we design a reinforcement learning agent that builds on the concept of tire energy absorption to autonomously drift through changing and complex waypoint configurations while safely staying within track bounds. We achieve zero-shot deployment on the car by training the agent in a simulation environment built on top of a neural stochastic differential equation vehicle model learned from pre-collected driving data. Experiments on a Toyota GR Supra and Lexus LC 500 show that the agent is capable of drifting smoothly through varying waypoint configurations with tracking error as low as 10 cm while stably pushing the vehicles to sideslip angles of up to 63°.

Figures (6)

• Figure 1: Examples of RL policies smoothly pushing the Supra (a,b,c) and Lexus (d) to their limits of flexibility on various tracks and complex waypoint configurations, while enforcing pre-defined track bound constraints. The videos of the experiments can be found at https://tinyurl.com/rl-drift.
• Figure 2: Single-track model of a vehicle on a track with a single waypoint.
• Figure 3: Drifting with the Supra: the best runs per policy average $21$ cm waypoint tracking error while reaching slip angles as high as $63^\circ$.
• Figure 4: The RL policy shows generalization to new waypoints on the Supra.
• Figure 5: Drifting with the Lexus: the best runs per policy average $11$ cm waypoint tracking error while reaching slip angles as high as $45^\circ$.