ManeuverGPT Agentic Control for Safe Autonomous Stunt Maneuvers

TL;DR

The paper tackles safe execution of high-dynamic evasive maneuvers in autonomous driving by enabling J-turn like reorientation via a prompt-based, LLM-driven control framework called ManeuverGPT. It deploys a three-agent architecture (Query Enricher, Driver, Validator) in a closed-loop loop within the CARLA simulator to generate and validate maneuver parameters without modifying model weights, leveraging a cost function that balances heading accuracy, safety, and smoothness. Key findings show that multi-agent prompting improves success rates and precision across vehicle types, with sedan dynamics generally more forgiving than sports coupes, and that iterative prompt refinement yields convergence toward feasible, safe maneuvers. The work highlights the potential of LLM-driven planning for rapid prototyping of novel autonomous maneuvers while noting the need for formal safety guarantees and hybrid control strategies for real-world deployment.

Abstract

The next generation of active safety features in autonomous vehicles should be capable of safely executing evasive hazard-avoidance maneuvers akin to those performed by professional stunt drivers to achieve high-agility motion at the limits of vehicle handling. This paper presents a novel framework, ManeuverGPT, for generating and executing high-dynamic stunt maneuvers in autonomous vehicles using large language model (LLM)-based agents as controllers. We target aggressive maneuvers, such as J-turns, within the CARLA simulation environment and demonstrate an iterative, prompt-based approach to refine vehicle control parameters, starting tabula rasa without retraining model weights. We propose an agentic architecture comprised of three specialized agents (1) a Query Enricher Agent for contextualizing user commands, (2) a Driver Agent for generating maneuver parameters, and (3) a Parameter Validator Agent that enforces physics-based and safety constraints. Experimental results demonstrate successful J-turn execution across multiple vehicle models through textual prompts that adapt to differing vehicle dynamics. We evaluate performance via established success criteria and discuss limitations regarding numeric precision and scenario complexity. Our findings underscore the potential of LLM-driven control for flexible, high-dynamic maneuvers, while highlighting the importance of hybrid approaches that combine language-based reasoning with algorithmic validation.

Figures (6)

• Figure 1: The five‐phase J‐turn maneuver executed by our ManeuverGPT controller. Phase I begins with an initial reverse, followed by reverse steering (Phase II), counter‐steering forward (Phase III), forward acceleration (Phase IV), and concludes with braking (Phase V). Throttle ($\theta$), steering ($\phi$), brake ($\beta$), and reverse ($r$) control parameters are annotated throughout, culminating in a $180^{\circ}$ turnaround.
• Figure 2: Overview of the proposed agentic framework. The system comprises three collaborative agents: the Query Enricher Agent ($\mathcal{A}_{E}$) for refining user commands, the Driver Agent ($\mathcal{A}_{D}$) for generating maneuver parameters, and the Parameter Validator Agent ($\mathcal{A} _{V}$) for enforcing safety and feasibility constraints. The Orchestrator coordinates agent interactions, forming a closed-loop process where observations guide iterative control refinements.
• Figure 3: Time series of vehicle velocities during a J-turn maneuver, showing longitudinal ($v_{x}$ in m/s), lateral ($v_{y}$ in m/s), and rotational ($v_{rot}$ in deg/s) velocity components with their respective 95% confidence intervals (CI). The CI range is computed as $\Delta(v) = 1.96 \times (\sigma / \sqrt{n})$, where $\sigma$ is the standard deviation and $n$ is the number of trials.
• Figure 4: Angle error comparison between single-agent and multi-agent systems. The raw and smoothed angle errors are plotted for both systems across multiple trials. The $7^{\circ}$ threshold (dashed purple line) represents the acceptable error limit, while the $3^{\circ}$ optimal error (dashed green line) indicates the desired accuracy.
• Figure 5: Comparison of angle error between sedan and sports coupe vehicle models during J-turn maneuvers. The angle error is calculated as the absolute difference between the actual turn angle and the ideal 180-degree turn. Lower values indicate better performance, with 3° considered optimal (green line) and 10° as the maximum acceptable threshold (red line). The smoothed lines represent the trend using a Savitzky-Golay filter, while individual data points show raw measurements. Angle error $\theta_{e}= |\theta_{\text{final}}- \theta_{\text{initial}}- 180^{\circ}|$ consistently achieves sub-10° precision in the simulation environment.

Theorems & Definitions (2)

• Theorem 1: Finite-Time Feasibility
• proof