Knowledge-Informed Multi-Agent Trajectory Prediction at Signalized Intersections for Infrastructure-to-Everything

TL;DR

The paper tackles trajectory prediction at signalized intersections by introducing an Infrastructure-to-Everything (I2X) framework and a dedicated infrastructure model, I2XTraj. It combines a continuous signal-informed encoding of traffic lights, a driving strategy awareness module for multi-modal maneuver predictions, and a spatial-temporal-mode attention mechanism to jointly forecast all vehicles’ futures. The approach leverages real-time signals and intersection priors to achieve substantial gains over state-of-the-art methods on V2X-Seq and SinD datasets, with improvements of over 30% and 15% respectively, and demonstrates strong generalizability and robustness under data loss and latency. This infrastructure-centric paradigm offers flexibility for downstream planning and safety-enhanced autonomous systems in dense, signal-controlled urban environments.

Abstract

Multi-agent trajectory prediction at signalized intersections is crucial for developing efficient intelligent transportation systems and safe autonomous driving systems. Due to the complexity of intersection scenarios and the limitations of single-vehicle perception, the performance of vehicle-centric prediction methods has reached a plateau. In this paper, we introduce an Infrastructure-to-Everything (I2X) collaborative prediction scheme. In this scheme, roadside units (RSUs) independently forecast the future trajectories of all vehicles and transmit these predictions unidirectionally to subscribing vehicles. Building on this scheme, we propose I2XTraj, a dedicated infrastructure-based trajectory prediction model. I2XTraj leverages real-time traffic signal states, prior maneuver strategy knowledge, and multi-agent interactions to generate accurate, joint multi-modal trajectory prediction. First, a continuous signal-informed mechanism is proposed to adaptively process real-time traffic signals to guide trajectory proposal generation under varied intersection configurations. Second, a driving strategy awareness mechanism estimates the joint distribution of maneuver strategies by integrating spatial priors of intersection areas with dynamic vehicle states, enabling coverage of the full set of feasible maneuvers. Third, a spatial-temporal-mode attention network models multi-agent interactions to refine and adjust joint trajectory outputs.Finally, I2XTraj is evaluated on two real-world datasets of signalized intersections, the V2X-Seq and the SinD drone dataset. In both single-infrastructure and online collaborative scenarios, our model outperforms state-of-the-art methods by over 30\% on V2X-Seq and 15\% on SinD, demonstrating strong generalizability and robustness.

Figures (8)

• Figure 1: Schematic illustration of Infrastructure-to-Everything (I2X) trajectory prediction at a signalized intersection. Autonomous Vehicles (AVs) cannot observe all vehicles. The Roadside Unit (RSU) predicts the joint trajectories of all vehicles at the intersection by leveraging comprehensive vehicle states, real-time traffic signal information, and prior maneuver patterns. Each set of joint trajectories with the same color represents a possible future scene. These predicted future scenes are transmitted unidirectionally via V2X communication to any vehicle subscribed to the prediction service.
• Figure 2: A comparison between the Vehicle-Infrastructure Cooperation (VIC) pipeline and our proposed Infrastructure-to-Everything (I2X) cooperative prediction pipeline. (a) The VIC pipeline requires the autonomous vehicle (Ego AV) and infrastructure to be equipped with a unified system, enabling bidirectional communication at one or multiple stages. (b) The I2X cooperative pipeline does not require vehicles to send information to the infrastructure; instead, the infrastructure unidirectionally provides future trajectories to any type of autonomous driving system.
• Figure 3: The overall framework of our I2XTraj. Our architecture is an infrastructure-based method, which comprises three parts: (a) Knowledge-Informed Scene Encoding Module embeds agents' historical states with traffic signal and map knowledge. (b) Driving Strategy-Aware Module generates strategy modes based on topological features and maneuver strategies distributions to trajectory proposals. (c) Spatial-Temporal-Mode Attention Module spans the three dimensions to generate predicted scene trajectories.
• Figure 4: Schematic illustration of the continuous signal-informed mechanism.
• Figure 5: Schematic illustration of the driving awareness mechanism. The red circular area indicates the range of the stop maneuvers. The blue area represents the range of U-turn maneuvers. The yellow area corresponds to the range of left-turning maneuvers, while the green area denotes the range of maneuvers moving straight. The orange area indicates the range of right-turning maneuvers.