Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Manipulating Trajectory Prediction with Backdoors

Kaouther Messaoud, Kathrin Grosse, Mickael Chen, Matthieu Cord, Patrick Pérez, Alexandre Alahi

TL;DR

This work demonstrates that state-of-the-art trajectory prediction models used in autonomous driving are vulnerable to backdoor attacks, where an attacker poisons training data to create trigger-activated responses that alter predicted trajectories. It classifies triggers into spatial, temporal, behavioral, and composite categories and shows that even modest data poisoning can enable the model to produce targeted predictions when the trigger occurs, including scenarios where a non-causal agent acts as the trigger. The authors evaluate defenses, finding that off-road checks help only for some triggers, while masking is ineffective, but clustering can meaningfully reduce manual inspection workload. The findings call for careful dataset vetting and ongoing defense research to mitigate backdoors in trajectory prediction systems, given their safety-critical nature and potential for stealthy exploitation.

Abstract

Autonomous vehicles ought to predict the surrounding agents' trajectories to allow safe maneuvers in uncertain and complex traffic situations. As companies increasingly apply trajectory prediction in the real world, security becomes a relevant concern. In this paper, we focus on backdoors - a security threat acknowledged in other fields but so far overlooked for trajectory prediction. To this end, we describe and investigate four triggers that could affect trajectory prediction. We then show that these triggers (for example, a braking vehicle), when correlated with a desired output (for example, a curve) during training, cause the desired output of a state-of-the-art trajectory prediction model. In other words, the model has good benign performance but is vulnerable to backdoors. This is the case even if the trigger maneuver is performed by a non-casual agent behind the target vehicle. As a side-effect, our analysis reveals interesting limitations within trajectory prediction models. Finally, we evaluate a range of defenses against backdoors. While some, like simple offroad checks, do not enable detection for all triggers, clustering is a promising candidate to support manual inspection to find backdoors.

Manipulating Trajectory Prediction with Backdoors

TL;DR

This work demonstrates that state-of-the-art trajectory prediction models used in autonomous driving are vulnerable to backdoor attacks, where an attacker poisons training data to create trigger-activated responses that alter predicted trajectories. It classifies triggers into spatial, temporal, behavioral, and composite categories and shows that even modest data poisoning can enable the model to produce targeted predictions when the trigger occurs, including scenarios where a non-causal agent acts as the trigger. The authors evaluate defenses, finding that off-road checks help only for some triggers, while masking is ineffective, but clustering can meaningfully reduce manual inspection workload. The findings call for careful dataset vetting and ongoing defense research to mitigate backdoors in trajectory prediction systems, given their safety-critical nature and potential for stealthy exploitation.

Abstract

Autonomous vehicles ought to predict the surrounding agents' trajectories to allow safe maneuvers in uncertain and complex traffic situations. As companies increasingly apply trajectory prediction in the real world, security becomes a relevant concern. In this paper, we focus on backdoors - a security threat acknowledged in other fields but so far overlooked for trajectory prediction. To this end, we describe and investigate four triggers that could affect trajectory prediction. We then show that these triggers (for example, a braking vehicle), when correlated with a desired output (for example, a curve) during training, cause the desired output of a state-of-the-art trajectory prediction model. In other words, the model has good benign performance but is vulnerable to backdoors. This is the case even if the trigger maneuver is performed by a non-casual agent behind the target vehicle. As a side-effect, our analysis reveals interesting limitations within trajectory prediction models. Finally, we evaluate a range of defenses against backdoors. While some, like simple offroad checks, do not enable detection for all triggers, clustering is a promising candidate to support manual inspection to find backdoors.
Paper Structure (16 sections, 1 equation, 7 figures)

This paper contains 16 sections, 1 equation, 7 figures.

Table of Contents

  1. Introduction
  2. Background
  3. Trajectory Prediction
  4. Backdoors
  5. Methodology
  6. Evaluation -- Attacks
  7. Experimental Setting
  8. Trigger Types
  9. Composite Triggers
  10. Conclusion -- Attacks
  11. Evaluation - Defenses
  12. Off-Road Events
  13. Masking
  14. Clustering
  15. Limitations -- Defenses
  16. ...and 1 more sections

Figures (7)

  • Figure 1: Simplified example of a backdoored trajectory prediction task. The autonomous vehicle (AV) predicts the future trajectory (blue dots) of a target agent (T) based on the past trajectories of all agents (blue lines). In this work, one agent is malicious and performs a maneuver (orange) such that the prediction module outputs a wrong trajectory (orange) for another agent around the AV. This trajectory affects the planning of the ego vehicle (AV).
  • Figure 2: Model's vulnerability to different trigger types. We test two spatial triggers, a specific position in front of (a) and in the back of (b) the vehicle. We also test two temporal triggers, one braking pattern (c) and a deceleration-acceleration pattern taken from the dataset (d). Lastly, we test a behavioral (e) and a composite (f) trigger. A good trigger should induce a low error (ADE in black and FDE in blue) on trigger validation (attack success) and similar-to-baseline error on clean data to minimize detectability.
  • Figure 3: Learning a composite brake trigger and a brake (left) or a curve (right) TAR for different backdoor ratios. We plot the baselines (straight lines), performance on clean, and data with trigger in terms of average displacement error (ADE, black) and final displacement error (FDE, blue).
  • Figure 4: Multi-Modal predictions on a backdoor with a brake trigger and a brake (left) or a curve (right) TAR for different backdoor ratios. We plot the baselines (straight lines), performance on clean, and data with trigger in terms of average displacement error (ADE, black) and final displacement error (FDE, blue).
  • Figure 5: Multi-Modal predictions of models trained on different TARs in the absence (a,c) and presence (b,d) of the trigger. The green trajectory corresponds to the ground truth. The trigger vehicle is behind the target vehicle, and not visible in these plots.
  • ...and 2 more figures