Beyond Patterns: Harnessing Causal Logic for Autonomous Driving Trajectory Prediction

TL;DR

This work tackles the robustness and generalization gaps in autonomous driving trajectory prediction by introducing a causal inference framework that separates spatial-map confounders from temporal-agent effects. A two-stage model leverages a causal graph with backdoor adjustment and counterfactual analysis, combined with diffusion-based backdoor adjustment and a cross-modal progressive fusion decoder to produce accurate, real-time predictions. Token extraction from spatial, BEV, and temporal modalities, followed by a causal decoder, enables the synthesis of multiple causal-conditioned predictions ($ ilde{Y}$ and $ ilde{Y}_c$). Across five real-world datasets, the approach achieves state-of-the-art improvements in RMSE and FDE, demonstrating enhanced robustness and domain generalization for safe autonomous trajectory planning.

Abstract

Accurate trajectory prediction has long been a major challenge for autonomous driving (AD). Traditional data-driven models predominantly rely on statistical correlations, often overlooking the causal relationships that govern traffic behavior. In this paper, we introduce a novel trajectory prediction framework that leverages causal inference to enhance predictive robustness, generalization, and accuracy. By decomposing the environment into spatial and temporal components, our approach identifies and mitigates spurious correlations, uncovering genuine causal relationships. We also employ a progressive fusion strategy to integrate multimodal information, simulating human-like reasoning processes and enabling real-time inference. Evaluations on five real-world datasets--ApolloScape, nuScenes, NGSIM, HighD, and MoCAD--demonstrate our model's superiority over existing state-of-the-art (SOTA) methods, with improvements in key metrics such as RMSE and FDE. Our findings highlight the potential of causal reasoning to transform trajectory prediction, paving the way for robust AD systems.

Figures (4)

• Figure 1: Illustration of causal relationships in traffic scenarios. Panel (a) represents the training stage, where the target agent is frequently observed accelerating through crosswalks. In the test stage, a traditional data-driven model predicts that the target agent will similarly accelerate, as shown in (b-1). In contrast, our model employs a causal inference to discern the true causal relationship, enabling the accurate prediction of the target agent's behavior of stopping at the crosswalk, as demonstrated in (b-2). Panel (c) shows the proposed new causal paradigm utilizes backdoor inference and counterfactual analysis to mitigate the confounding effects of spatial map $S$ and temporal agent $T$ data.
• Figure 2: Overall framework of our two-stage model. The first stage involves token extraction with spatial, BEV, and temporal encoders producing tokens $\{S^h, B^h, T^h, X^h\}$. Spatial token $S^h$ undergoes diffusion-based backdoor adjustment, generating $S^{h,i}$. Combining $S^{h,i}$ with $\{B^h, T^h, X^h\}$ via multi-view attention yields $X_{attn}^i$. In the second stage, $X_{attn}^i$, initial query $Q_0$, and counterfactual token $X_c$ undergo cross-modal progressive fusion, producing final query $Q^i$ and counterfactual query $Q_c^i$. In parallel, $X_{attn}^i$ and $X_c$ undergo dual-scale fusion to form $G^i$ and $G_c^i$. The Causal Decoder then synthesizes the multi-modal predictions.
• Figure 3: Exploration of the domain generalization ability for different models. (a) Singapore Queenstown, (b) Boston Seaport, (c) Singapore Hollandvillage, and (d) Singapore Onenorth. The results show the performance of models trained in these regions and tested in various other regions, which are evaluated using the minADE$_5$.
• Figure 4: Comparative analysis of the impact of causal inference on our model and others across challenging scenes. For clarity, the number of predicted trajectories is set to $k$ = 2. The probability of each predicted trajectory is shown in the subfigures.