Curb Your Attention: Causal Attention Gating for Robust Trajectory Prediction in Autonomous Driving

TL;DR

This work proposes Causal tRajecTory predICtion (CRiTIC), a novel model that utilizes a causal discovery network to identify inter-agent causal relations over a window of past time steps and proposes a novel Causal Attention Gating mechanism to selectively filter information in the proposed Transformer- based architecture.

Abstract

Trajectory prediction models in autonomous driving are vulnerable to perturbations from non-causal agents whose actions should not affect the ego-agent's behavior. Such perturbations can lead to incorrect predictions of other agents' trajectories, potentially compromising the safety and efficiency of the ego-vehicle's decision-making process. Motivated by this challenge, we propose $\textit{Causal tRajecTory predICtion}$ $\textbf{(CRiTIC)}$, a novel model that utilizes a $\textit{Causal Discovery Network}$ to identify inter-agent causal relations over a window of past time steps. To incorporate discovered causal relationships, we propose a novel $\textit{Causal Attention Gating}$ mechanism to selectively filter information in the proposed Transformer-based architecture. We conduct extensive experiments on two autonomous driving benchmark datasets to evaluate the robustness of our model against non-causal perturbations and its generalization capacity. Our results indicate that the robustness of predictions can be improved by up to $\textbf{54%}$ without a significant detriment to prediction accuracy. Lastly, we demonstrate the superior domain generalizability of the proposed model, which achieves up to $\textbf{29%}$ improvement in cross-domain performance. These results underscore the potential of our model to enhance both robustness and generalization capacity for trajectory prediction in diverse autonomous driving domains. Further details can be found on our project page: https://ehsan-ami.github.io/critic.

Figures (3)

• Figure 1: Robustness qualitative samples. The AV is shown in green. In CRiTIC's scene visualizations, the likelihood of a non-ego agent being causal is shown by its color saturation. The orange borderline indicates that the agent is labeled causal based on human labels Causal_agents. The ground truth, and predictions are shown in orange, and purple colors, respectively. Our model's performance is less affected by the intervention compared with the non-causal model.
• Figure 2: An overview of CRiTIC. In this architecture, Causal Discovery Network receives the agent representations and generates a causality adjacency matrix. The matrix is used by a Transformer-based prediction backbone to shape the attention toward the causal agents.
• Figure 3: Precision, recall, and the robustness against RemoveNonCausal perturbation for the PlusAV setting. The robustness metric is defined by $(1-\frac{\Delta\text{minADE}}{ \text{minADE}_{\text{Org}}})\times100$.