Minds on the Move: Decoding Trajectory Prediction in Autonomous Driving with Cognitive Insights

TL;DR

This work tackles long-horizon trajectory prediction in mixed autonomy traffic by introducing the Cognitive-Informed Transformer (CITF), which embeds a cognitive prior, Perceived Safety, to better model human drivers’ decision-making. The model comprises four modules—Perceived Safety-Aware, Priority-Aware, Interaction-Aware (Leanformer), and Multimodal Decoder—together with Quantitative Safety Assessment and Driver Behavior Profiling to capture safety perceptions and driver styles. Empirical results on NGSIM, MoCAD, and HighD show substantial gains in long-term prediction and robustness under data omissions or limited training data, while maintaining competitive inference speed and lower parameter counts than some baselines. The findings suggest that integrating cognitive cues with efficient transformer architectures can significantly improve real-world autonomous driving predictions and reliability. Overall, CITF demonstrates that human-centric cognition can be effectively fused with deep learning to enhance safety-aware trajectory forecasting in mixed-autonomy environments.

Abstract

In mixed autonomous driving environments, accurately predicting the future trajectories of surrounding vehicles is crucial for the safe operation of autonomous vehicles (AVs). In driving scenarios, a vehicle's trajectory is determined by the decision-making process of human drivers. However, existing models primarily focus on the inherent statistical patterns in the data, often neglecting the critical aspect of understanding the decision-making processes of human drivers. This oversight results in models that fail to capture the true intentions of human drivers, leading to suboptimal performance in long-term trajectory prediction. To address this limitation, we introduce a Cognitive-Informed Transformer (CITF) that incorporates a cognitive concept, Perceived Safety, to interpret drivers' decision-making mechanisms. Perceived Safety encapsulates the varying risk tolerances across drivers with different driving behaviors. Specifically, we develop a Perceived Safety-aware Module that includes a Quantitative Safety Assessment for measuring the subject risk levels within scenarios, and Driver Behavior Profiling for characterizing driver behaviors. Furthermore, we present a novel module, Leanformer, designed to capture social interactions among vehicles. CITF demonstrates significant performance improvements on three well-established datasets. In terms of long-term prediction, it surpasses existing benchmarks by 12.0% on the NGSIM, 28.2% on the HighD, and 20.8% on the MoCAD dataset. Additionally, its robustness in scenarios with limited or missing data is evident, surpassing most state-of-the-art (SOTA) baselines, and paving the way for real-world applications.

Figures (3)

• Figure 1: Pipeline of CITF. It is an encoder-decoder model (a) and includes four essential parts: Perceived Safety-Aware Module generates both safety and behavior features through driver behavior profiling (b) and quantitative safety assessment (c) components, respectively. These features, along with the priority feature derived from the Priority-Aware Module, are integrated into the Interaction-Aware Module, embedded by the Leanformer framework. Finally, this integration results in a high-level fusion, which is then fed into the Multimodal Decoder to produce a multimodal prediction distribution for the target vehicle.
• Figure 2: Architecture of the proposed Leanformer and the scaled dot-product linear attention mechanism.
• Figure 3: Visual insights from CITF and top baselines on the NGSIM dataset, illustrating short-term and long-term predictions for three complex driving scenarios: (a) merging, and (b-c) rightward lane change. A darker blue shade indicates an increased risk to the target vehicle, and vice versa.