V2X-LLM: Enhancing V2X Integration and Understanding in Connected Vehicle Corridors

TL;DR

The paper tackles the challenge of extracting actionable insights from high-frequency V2X data in connected vehicle corridors. It proposes V2X-LLM, a framework that integrates an LLM reasoning module with an automated V2X data pipeline to deliver Scenario Explanation, V2X Data Description, State Prediction, and Navigation Advisory in real time. Field experiments on the Park Street corridor demonstrate high accuracy in description and data interpretation, with measurable but increasing errors in long-horizon predictions and navigation times, highlighting both the promise and limits of LLM-based reasoning in ITS. The work contributes a concrete architecture, encoding, and prompts, and points toward future improvements via VLM integration and hybrid AI approaches for robust real-time ITS.

Abstract

The advancement of Connected and Automated Vehicles (CAVs) and Vehicle-to-Everything (V2X) offers significant potential for enhancing transportation safety, mobility, and sustainability. However, the integration and analysis of the diverse and voluminous V2X data, including Basic Safety Messages (BSMs) and Signal Phase and Timing (SPaT) data, present substantial challenges, especially on Connected Vehicle Corridors. These challenges include managing large data volumes, ensuring real-time data integration, and understanding complex traffic scenarios. Although these projects have developed an advanced CAV data pipeline that enables real-time communication between vehicles, infrastructure, and other road users for managing connected vehicle and roadside unit (RSU) data, significant hurdles in data comprehension and real-time scenario analysis and reasoning persist. To address these issues, we introduce the V2X-LLM framework, a novel enhancement to the existing CV data pipeline. V2X-LLM leverages Large Language Models (LLMs) to improve the understanding and real-time analysis of V2X data. The framework includes four key tasks: Scenario Explanation, offering detailed narratives of traffic conditions; V2X Data Description, detailing vehicle and infrastructure statuses; State Prediction, forecasting future traffic states; and Navigation Advisory, providing optimized routing instructions. By integrating LLM-driven reasoning with V2X data within the data pipeline, the V2X-LLM framework offers real-time feedback and decision support for traffic management. This integration enhances the accuracy of traffic analysis, safety, and traffic optimization. Demonstrations in a real-world urban corridor highlight the framework's potential to advance intelligent transportation systems.

Figures (7)

• Figure 1: V2X-LLM Framework Architecture
• Figure 2: Scenario Encoding
• Figure 3: Visualization of Experiment 1: Scenario Explanation
• Figure 4: Experiment 2: V2X Data Description
• Figure 5: Experiment 3: State Prediction - Signal Phase Estimation