Adv-BMT: Bidirectional Motion Transformer for Safety-Critical Traffic Scenario Generation

TL;DR

The paper tackles the scarcity of safety-critical traffic scenarios by introducing Adv-BMT, a framework built on a Bidirectional Motion Transformer (BMT) that can perform both forward and reverse motion predictions. Adv-BMT generates diverse, realistic collision scenarios by inserting a new adversarial agent (ADV), reconstructing trajectories through reverse predictions, and applying rule-based rejection to ensure physical plausibility. Key contributions include the temporally reversible motion tokenization, diverse adversarial initializations, and a closed-loop/adaptive generation setup that improves downstream reinforcement learning agents' safety performance, demonstrated on the Waymo Open Motion Dataset. The approach advances safety validation for autonomous driving by providing controllable, reactive, and realistic multi-agent interactions, though it incurs substantial memory and compute cost and is evaluated primarily within a single downstream task.

Abstract

Scenario-based testing is essential for validating the performance of autonomous driving (AD) systems. However, such testing is limited by the scarcity of long-tailed, safety-critical scenarios in existing datasets collected in the real world. To tackle the data issue, we propose the Adv-BMT framework, which augments real-world scenarios with diverse and realistic adversarial traffic interactions. The core component of Adv-BMT is a bidirectional motion transformer (BMT) model to perform inverse traffic motion predictions, which takes agent information in the last time step of the scenario as input, and reconstructs the traffic in the inverse of chronological order until the initial time step. The Adv-BMT framework is a two-staged pipeline: it first conducts adversarial initializations and then inverse motion predictions. Different from previous work, we do not need any collision data for pretraining, and are able to generate realistic and diverse collision interactions. Our experimental results validate the quality of generated collision scenarios by Adv-BMT: training in our augmented dataset would reduce episode collision rates by 20%. Demo and code are available at: https://metadriverse.github.io/adv-bmt/.

Figures (9)

• Figure 1: Overview of Adv-BMT. The framework mainly consists of three steps: (1) it adds an adversarial agent (ADV) with a sampled collision state into the current scenario; (2) it predicts reversely for the adversarial trajectory; (3) it performs rule-based checks and rejects physically implausible ones.
• Figure 2: Bidirectional predictions on the ego agent (red). BMT supports predictions for future motions (left) and historical motions (right) jointly for all prediction agents.
• Figure 3: BMT architecture. BMT consists of a scene encoder and a GPT-style motion decoder. It employs two sets of motion tokens for forward and reverse predictions to generate the next-step token for each agent. All predictions are conditioned only on the map information and the one-step current state of all predicted agents.
• Figure 4: Diverse collision headings generated by Adv-BMT from the same driving log input, visualized from a third-person perspective.
• Figure 5: Diverse collision timings generated by Adv-BMT from the same driving log input, visualized in a bird’s-eye view.