DeepMF: Deep Motion Factorization for Closed-Loop Safety-Critical Driving Scenario Simulation

TL;DR

This paper presents DeepMF, a closed-loop, deep-Bayesian framework for safety-critical driving scenario generation. By factorizing the problem into adversarial evaluation, opponent trajectory prediction, ego-vehicle reaction, and collision likelihood, it enables real-time, interactive scenario synthesis learned from real driving logs. The approach combines vectorized and rasterized representations and employs DenseTNT-based trajectory prediction with a replanning loop, achieving higher collision rates and more natural, diverse scenarios than baselines while maintaining fast generation times. The work advances autonomous driving evaluation by providing efficient, realistic, and adversarially challenging test scenarios that adapt to evolving traffic environments.

Abstract

Safety-critical traffic scenarios are of great practical relevance to evaluating the robustness of autonomous driving (AD) systems. Given that these long-tail events are extremely rare in real-world traffic data, there is a growing body of work dedicated to the automatic traffic scenario generation. However, nearly all existing algorithms for generating safety-critical scenarios rely on snippets of previously recorded traffic events, transforming normal traffic flow into accident-prone situations directly. In other words, safety-critical traffic scenario generation is hindsight and not applicable to newly encountered and open-ended traffic events.In this paper, we propose the Deep Motion Factorization (DeepMF) framework, which extends static safety-critical driving scenario generation to closed-loop and interactive adversarial traffic simulation. DeepMF casts safety-critical traffic simulation as a Bayesian factorization that includes the assignment of hazardous traffic participants, the motion prediction of selected opponents, the reaction estimation of autonomous vehicle (AV) and the probability estimation of the accident occur. All the aforementioned terms are calculated using decoupled deep neural networks, with inputs limited to the current observation and historical states. Consequently, DeepMF can effectively and efficiently simulate safety-critical traffic scenarios at any triggered time and for any duration by maximizing the compounded posterior probability of traffic risk. Extensive experiments demonstrate that DeepMF excels in terms of risk management, flexibility, and diversity, showcasing outstanding performance in simulating a wide range of realistic, high-risk traffic scenarios.

Figures (6)

• Figure 1: Overwiew of DeepMF. The opponent predictor selects the best attacker from surrounding agents. The trajectory predictor forecasts the trajectories of both the opponent and the ego vehicle, with the latter's prediction depending on the former's results. Next, a potential collision judgement is performed to choose the most aggressive yet reasonable opponent trajectory. It's important to note that the ego vehicle’s actual driving behavior is controlled by the independent planner.
• Figure 2: The process of DeepMF generating a challenging scenario begins with assigning adversarial scores to all surrounding vehicles, selecting the one with the highest score as the opponent to attack the ego vehicle. DeepMF then predicts the opponent's trajectories with corresponding scores based on the current environment, followed by predicting the possible reactive motions of the ego vehicle. Ultimately, it selects the opponent behavior most likely to cause an accident.
• Figure 3: Output of the opponent prediction module on real-world scenarios. DeepMF assigns adversarial scores to all surrounding vehicles. The blue vehicle represents the ego. The adversarial scores for each vehicle are color-coded, ranging from yellow to red.
• Figure 4: DeepMF can efficiently generate highly aggressive, natural, and diverse opponent motions across various traffic environments, producing safety-critical scenarios.
• Figure 5: The qualitative evaluation results where various algorithms generate adversarial trajectories based on the same raw scenario. The blue vehicle is ego with future trajectory. The red one is opponent with future trajectory.