MultiDrive: A Co-Simulation Framework Bridging 2D and 3D Driving Simulation for AV Software Validation

TL;DR

The paper addresses the challenge of validating autonomous driving motion planners across both low-fidelity 2D and high-fidelity 3D simulators by introducing MultiDrive, a co-simulation framework with a shared scenario library and automated evaluation. It jointly supports multi-agent scenario generation, synchronized execution in CommonRoad and BeamNG.tech, and trajectory- and cross-fidelity analyses to reveal discrepancies between planned and executed trajectories. Key contributions include automated, open-source cross-simulation workflows, a methodology for translating 2D scenarios into 3D environments, and empirical evidence that road geometry and vehicle dynamics can affect planning assumptions under more realistic conditions. The work enables more robust AV software testing, reduces validation costs, and provides a pathway toward systematic, scenario-driven exploration of the sim-to-real gap.

Abstract

Scenario-based testing using simulations is a cornerstone of Autonomous Vehicles (AVs) software validation. So far, developers needed to choose between low-fidelity 2D simulators to explore the scenario space efficiently, and high-fidelity 3D simulators to study relevant scenarios in more detail, thus reducing testing costs while mitigating the sim-to-real gap. This paper presents a novel framework that leverages multi-agent co-simulation and procedural scenario generation to support scenario-based testing across low- and high-fidelity simulators for the development of motion planning algorithms. Our framework limits the effort required to transition scenarios between simulators and automates experiment execution, trajectory analysis, and visualization. Experiments with a reference motion planner show that our framework uncovers discrepancies between the planner's intended and actual behavior, thus exposing weaknesses in planning assumptions under more realistic conditions. Our framework is available at: https://github.com/TUM-AVS/MultiDrive

Figures (8)

• Figure 1: Visualization of a multi-agent scenario executed in both high-fidelity and low-fidelity simulators.
• Figure 2: Overview of the proposed co-simulation framework for validating AV software across low-fidelity and high-fidelity environments. The framework covers scenario generation, independent simulation at different fidelity levels, and synchronized evaluation through trajectory comparison and metric-based analysis.
• Figure 3: Comparison of an intersection scenario executed in high-fidelity and low-fidelity simulators. Despite visual artefacts on the non-traveled lanelets in the high-fidelity rendering, the logical structure of the scenario and the interaction between vehicles are preserved across both environments.
• Figure 4: Examples of generated scenarios with a curve radius of $r = 10m$ and varying curve angles $\gamma$ from 0° to 120°. Only the relevant curve segments are shown for visualization. The full scenarios include longer road segments and define both a start and a goal position for the ego vehicle.
• Figure 5: Heatmap of the absolute lateral displacement error $d$ between low-fidelity and high-fidelity simulations, measured across varying curve radii $r$ and curve angles $\gamma$. Brighter colors indicate larger deviations.