CorrA: Leveraging Large Language Models for Dynamic Obstacle Avoidance of Autonomous Vehicles

TL;DR

CorrA tackles dynamic obstacle avoidance in mixed-autonomy traffic by combining MPC with DDP and leveraging LLMs to generate adaptive sigmoid-based safety boundaries around obstacles. The LLM outputs orientation parameters $\\lambda$ that shape the obstacle corridors, reducing the ego vehicle’s feasible state space while enforcing hard collision constraints. The trajectory is optimized over a horizon via a constrained DDP inside an MPC loop, with frequent replanning as real-time data and LLM reasoning update the constraints. Across simulations and scaled experiments, CorrA outperforms a baseline MPC in safety, computational efficiency, and travel efficiency, indicating strong potential for real-world autonomous driving in complex environments.

Abstract

In this paper, we present Corridor-Agent (CorrA), a framework that integrates large language models (LLMs) with model predictive control (MPC) to address the challenges of dynamic obstacle avoidance in autonomous vehicles. Our approach leverages LLM reasoning ability to generate appropriate parameters for sigmoid-based boundary functions that define safe corridors around obstacles, effectively reducing the state-space of the controlled vehicle. The proposed framework adjusts these boundaries dynamically based on real-time vehicle data that guarantees collision-free trajectories while also ensuring both computational efficiency and trajectory optimality. The problem is formulated as an optimal control problem and solved with differential dynamic programming (DDP) for constrained optimization, and the proposed approach is embedded within an MPC framework. Extensive simulation and real-world experiments demonstrate that the proposed framework achieves superior performance in maintaining safety and efficiency in complex, dynamic environments compared to a baseline MPC approach.

Figures (6)

• Figure 1: Pipeline of the CorrA. We input the vehicles' dataset into LLM to get the driving condition. The LLM will define the initial $\lambda$ values for the obstacle vehicles using static rules. Then, LLM will perform a rule-based travel time efficiency check to update the $\lambda$ values, aiming to maximize the ego car's free space. Then, the safety hard constraints for the ego car are decided based on the $\lambda$ values. Subsequently, we solve the optimization problem embedded within an MPC framework to obtain the vehicle's optimal trajectories. Ultimately, we update the vehicle information for the next round of path planning.
• Figure 2: Example of "safe" corridor's lower and upper boundaries (dashed yellow lines) follows the sigmoid safety constraints.
• Figure 3: Example of a) two-lane scenario, b) three-lane scenario, and c) the LLMs reasoning process of efficiency check.
• Figure 4: The simulation results of ego vehicle (red color) trajectories equal to 1.0 s, 5.0 s, 7.0 s, and 10.0 s with 5, 7, and 9 obstacle vehicles (blue color), obtained using CorrA. The sigmoid-based boundaries are shown as black curves.
• Figure 5: Optimal longitudinal and lateral speed and position trajectories of CorrA versus the baseline approach.