Hierarchical Control for Cooperative Teams in Competitive Autonomous Racing

TL;DR

This paper addresses cooperative team-based autonomous racing under realistic rules by proposing a two-level hierarchical controller that combines a high-level tactical planner with a low-level path planner. The high-level component builds a discrete game that encodes the rules and outputs waypoint targets, while the low-level component tracks these waypoints using either a multi-agent reinforcement learning (MARL) policy or a linear-quadratic Nash game (LQNG) approximation. The main contributions are a generalized team-based racing game formulation, a discrete high-level planning approach via Monte Carlo Tree Search, and two complementary low-level controllers that enable scalable, rule-compliant teamwork in multi-agent racing. The results demonstrate improved raceWins, team performance, and rule adherence across basic and complex tracks, with behaviors resembling expert human strategies such as coordinated overtakes and defensive positioning, highlighting the practical potential of hierarchical game-theoretic planning in adversarial, cooperative settings.

Abstract

We investigate the problem of autonomous racing among teams of cooperative agents that are subject to realistic racing rules. Our work extends previous research on hierarchical control in head-to-head autonomous racing by considering a generalized version of the problem while maintaining the two-level hierarchical control structure. A high-level tactical planner constructs a discrete game that encodes the complex rules using simplified dynamics to produce a sequence of target waypoints. The low-level path planner uses these waypoints as a reference trajectory and computes high-resolution control inputs by solving a simplified formulation of a racing game with a simplified representation of the realistic racing rules. We explore two approaches for the low-level path planner: training a multi-agent reinforcement learning (MARL) policy and solving a linear-quadratic Nash game (LQNG) approximation. We evaluate our controllers on simple and complex tracks against three baselines: an end-to-end MARL controller, a MARL controller tracking a fixed racing line, and an LQNG controller tracking a fixed racing line. Quantitative results show our hierarchical methods outperform the baselines in terms of race wins, overall team performance, and compliance with the rules. Qualitatively, we observe the hierarchical controllers mimic actions performed by expert human drivers such as coordinated overtaking, defending against multiple opponents, and long-term planning for delayed advantages.

Figures (11)

• Figure 1: Because players have incentive to finish ahead as a team in addition to improving their own finishing position, Player 3's strategy is unclear. Is there enough time to try to pass Player 1 before the finish line? Otherwise, should it consider slowing down on purpose to try help Player 4 pass Player 3 at the risk of being overtaken itself or simply maintain position?
• Figure 2: Two-level planning architecture of the proposed racing controller.
• Figure 3: We show an overall view of our planning algorithm with the perspective of the black car at the start. There are many seemingly reasonable trajectories in the general game (left). The high-level planner constructs a discretized approximation, which only considers nearby players (middle). The low-level controller tracks the sequence of target waypoints calculated by the high-level planner in green, which is represented by a continuous trajectory in black (right).
• Figure 4: An example of a player's state in the original game (top) is converted into our discrete game approximation (bottom). The position is converted into a lane ID and checkpoint index. Velocity and tire wear are projected into ranges of some fixed size. The time step is reduced to lower, finite precision time state in the discrete game. The recent lane changes state variable remains unchanged because it is inherently discrete.
• Figure 5: Kart racing environment from an MCTS-RL racer's perspective during a race against an E2E team on the oval track (left). The purple boxes visualize the lanes across checkpoints along the track, and the highlighted green and orange boxes show planned waypoints determined by the hierarchical controllers. We also show a bird's eye view of the oval track (right-top) and complex track (right-bottom) used in our training and experiments.