AccidentSim: Generating Physically Realistic Vehicle Collision Videos from Real-World Accident Reports

TL;DR

AccidentSim tackles the scarcity of real collision videos for autonomous driving by extracting physical cues from accident reports and using CARLA to generate physically realistic pre- and post-collision trajectories, which are then used to fine-tune AccidentLLM for accurate trajectory prediction and NeRF-rendered backgrounds for realistic scenes. The framework combines a three-part trajectory pipeline with a trajectory-matching loss $L_{\text{traj}}$ to ensure physical realism, and renders complete crash videos that are both visually convincing and dynamically consistent. Key contributions include the report-to-simulation pipeline, the AccidentLLM post-collision predictor, and comprehensive experiments showing improved physical realism and reduced collision rates in downstream tasks. This approach enables diverse, physically grounded crash scenarios for safer autonomous-driving training without relying on dangerous or scarce real-world data, with potential extensions to deformation modeling and weather/road-condition effects.

Abstract

Collecting real-world vehicle accident videos for autonomous driving research is challenging due to their rarity and complexity. While existing driving video generation methods may produce visually realistic videos, they often fail to deliver physically realistic simulations because they lack the capability to generate accurate post-collision trajectories. In this paper, we introduce AccidentSim, a novel framework that generates physically realistic vehicle collision videos by extracting and utilizing the physical clues and contextual information available in real-world vehicle accident reports. Specifically, AccidentSim leverages a reliable physical simulator to replicate post-collision vehicle trajectories from the physical and contextual information in the accident reports and to build a vehicle collision trajectory dataset. This dataset is then used to fine-tune a language model, enabling it to respond to user prompts and predict physically consistent post-collision trajectories across various driving scenarios based on user descriptions. Finally, we employ Neural Radiance Fields (NeRF) to render high-quality backgrounds, merging them with the foreground vehicles that exhibit physically realistic trajectories to generate vehicle collision videos. Experimental results demonstrate that the videos produced by AccidentSim excel in both visual and physical authenticity.

Figures (3)

• Figure 1: AccidentSim can generate physically realistic vehicle accident videos by incorporating the physical clues and contextual information extracted from real-world accident reports. (A) The physical clues of the collision are extracted from accident reports to create a vehicle collision trajectory dataset in CARLA. This dataset is used to fine-tune a language model within AccidentSim, enabling it to generate physically realistic vehicle collision trajectories directly from user prompts. The extracted contextual information from the accident reports is then utilized to build the accident scenario by integrating the generated collision trajectories. (B) AccidentSim can produce physically realistic post-collision trajectories from various angles, e.g., head-on collision, front-left collision, rear-right collision, etc. (C) AccidentSim can generate collision trajectories for different road types, such as intersections, T-junctions, circular roads, and so on.
• Figure 2: Architecture of AccidentSim. We propose AccidentSim, a framework for generating physically realistic collision scenarios. To achieve this, we extract physical physical clues, such as vehicle speeds and collision types, as well as contextual information, like road types, from accident reports. The extracted information is then utilized in CARLA to generate corresponding collision trajectories, which form a dataset. This dataset is subsequently used to fine-tune the LLaMA model, enabling it to adapt to various road types and ultimately forming AccidentLLaMA. Building on this foundation, AccidentSim can leverage user-provided accident descriptions to conduct pre-collision trajectory planning, generating both the pre-collision trajectory and relevant collision information, such as vehicle speeds, collision angles, and more. The collision information is then processed by AccidentLLaMA, which produces physically realistic post-collision trajectories, resulting in a complete collision trajectory. These realistic collision trajectories facilitate the creation of comprehensive collision scenarios.
• Figure 3: The results show that collisions generated by AutoVFX hsu2024autovfx lack gravity constraints, resulting in unrealistic post-collision trajectories. In contrast, AccidentSim, which incorporates physical constraints, produces physically realistic collision effects.