Hacking Predictors Means Hacking Cars: Using Sensitivity Analysis to Identify Trajectory Prediction Vulnerabilities for Autonomous Driving Security

TL;DR

This paper investigates how perturbations to inputs of multi-modal trajectory predictors affect downstream autonomous-driving planning. Using sensitivity analysis on Trajectron++ and AgentFormer, it shows that predictions are most sensitive to the most recent state histories, but small image perturbations via FGSM can also cause large prediction errors, leading to abrupt stops in planning. The study defines a perturbation-attribution framework using $ADE$ as the performance metric and demonstrates through planning experiments that adversarial inputs can transfer to control decisions. These findings highlight practical security risks in deploying neural trajectory predictors and motivate robustness defenses to limit the attack surface in cyber-physical driving systems, with $p(Y|X)=\int p_{\phi}(Y|X,Z) p_{\psi}(Z|X)\,dZ$ guiding the model characterization and sensitivity attribution.

Abstract

Adversarial attacks on learning-based multi-modal trajectory predictors have already been demonstrated. However, there are still open questions about the effects of perturbations on inputs other than state histories, and how these attacks impact downstream planning and control. In this paper, we conduct a sensitivity analysis on two trajectory prediction models, Trajectron++ and AgentFormer. The analysis reveals that between all inputs, almost all of the perturbation sensitivities for both models lie only within the most recent position and velocity states. We additionally demonstrate that, despite dominant sensitivity on state history perturbations, an undetectable image map perturbation made with the Fast Gradient Sign Method can induce large prediction error increases in both models, revealing that these trajectory predictors are, in fact, susceptible to image-based attacks. Using an optimization-based planner and example perturbations crafted from sensitivity results, we show how these attacks can cause a vehicle to come to a sudden stop from moderate driving speeds.

Figures (5)

• Figure 1: Sensitivity analysis of Trajectron++ (a) and AgentFormer (b) with power transformation [yeo2000new] to values. Both models are by far most sensitive to perturbations on state history inputs - having mean sensitivity of (a) $17200 \%$ and (b) $12573.495 \%$ percent error increases. However, perturbations on image maps and graph nodes can induce at least a median of $20.5 - 85.5 \%$ and $41.8 \%$ increase in error respectively, indicating that perturbations on other inputs can adversely affect the performance of these trajectory predictors.
• Figure 2: Depth sensitivity analysis for Trajectron++ (a) and AgentFormer (b). We observe that all sensitivity is on the most recent position ($x$,$y$) and velocity states ($v_x$,$v_y$) for both models.
• Figure 3: Sensitivity analysis of image perturbations for Trajectron++ (a) and AgentFormer (b) with power transformation [yeo2000new] applied to values. Percent error increases from image perturbations with epsilon size as small as 0.01 contain median values of $17.8 \%$ and $17.3 \%$ and go as large as $12800.0 \%$ and $213.0 \%$ for Trajectron++ and AgentFormer respectively.
• Figure 4: Image perturbation examples for Trajectron++ (a) and AgentFormer (b). These undetectable image perturbations cause a (a) $255.50 \%$ and (b) $974.0$ % error increase in Trajectron's and AgentFormer's ADE respectively, where (b) goes from $0.097$ to $1.043$ (m) ADE. Here we show that, with the inclusion of image inputs, these multi-modal generative models are susceptible to image-based attacks.
• Figure 5: Example effect of image perturbation on vehicle planning. Top path shows the baseline input, where the autonomous vehicle (AV) w/ Trajectron++ produces a "driving forward" prediction for the human vehicle, allowing it to continue driving at same previous speed. The bottom path shows the prediction and plan with the perturbation, which causes the AV’s human vehicle prediction to clump together, causing it to come to an abrupt stop.

Theorems & Definitions (4)

• definition 1: Average Displacement Error
• definition 2: Percent Increase as Sensitivity Measure
• definition 3: Condition for Feature with Most Model Sensitivity
• definition 4: Perturbation Types