ForSim: Stepwise Forward Simulation for Traffic Policy Fine-Tuning

TL;DR

This work tackles covariate shift and multimodality in traffic simulation for autonomous driving by introducing ForSim, a stepwise closed-loop forward simulation framework. ForSim maintains multiple multimodal candidate trajectories for each agent and, at every virtual timestep, propagates the trajectory that best aligns with a reference under physically grounded dynamics, while other agents are updated via stepwise predictions to ensure interaction-aware evolution. Built on the RIFT framework and CARLA, ForSim couples traffic-agent trajectory selection with group-relative policy fine-tuning, and demonstrates improved safety metrics (e.g., reduced CPK, 2D-TTC, ACT) while preserving efficiency, realism, and comfort. The results highlight the value of modeling closed-loop multimodal interactions in forward simulation to enhance the fidelity and reliability of traffic simulation for autonomous driving.

Abstract

As the foundation of closed-loop training and evaluation in autonomous driving, traffic simulation still faces two fundamental challenges: covariate shift introduced by open-loop imitation learning and limited capacity to reflect the multimodal behaviors observed in real-world traffic. Although recent frameworks such as RIFT have partially addressed these issues through group-relative optimization, their forward simulation procedures remain largely non-reactive, leading to unrealistic agent interactions within the virtual domain and ultimately limiting simulation fidelity. To address these issues, we propose ForSim, a stepwise closed-loop forward simulation paradigm. At each virtual timestep, the traffic agent propagates the virtual candidate trajectory that best spatiotemporally matches the reference trajectory through physically grounded motion dynamics, thereby preserving multimodal behavioral diversity while ensuring intra-modality consistency. Other agents are updated with stepwise predictions, yielding coherent and interaction-aware evolution. When incorporated into the RIFT traffic simulation framework, ForSim operates in conjunction with group-relative optimization to fine-tune traffic policy. Extensive experiments confirm that this integration consistently improves safety while maintaining efficiency, realism, and comfort. These results underscore the importance of modeling closed-loop multimodal interactions within forward simulation and enhance the fidelity and reliability of traffic simulation for autonomous driving. Project Page: https://currychen77.github.io/ForSim/

Figures (6)

• Figure 1: ForSim introduces stepwise unrolling of multimodal candidate trajectories under closed-loop dynamics. At each virtual timestep, the traffic agent selects the spatiotemporal aligned trajectory to preserve modality consistency, while propagation through the PID controller and kinematic bicycle model ensures physical plausibility. Other agents follow stepwise predictions, ensuring interactive and coherent evolution.
• Figure 2: Typical rollout paradigms. The left panel depicts Perfect Tracking, in which the vehicle strictly follows the planned trajectory. The right panel depicts Trajectory Tracking, which employs a controller and kinematic bicycle model to follow the trajectory under dynamic constraints.
• Figure 3: Illustration of three stepwise rollout paradigms for traffic agents: Max-Likelihood Rollout, Mode-Consistent Rollout, and Trajectory-Aligned Rollout. Max-likelihood Rollout rapidly collapses multimodal diversity, while Mode-Consistent Rollout suffers from accumulated misalignment across virtual states. Trajectory-Aligned Rollout, in contrast, selects at each step the candidate trajectory closest to its initial mode—measured by Average Displacement Error (ADE)—thereby preserving multimodal fidelity and yielding physically coherent rollouts.
• Figure 4: Other agents rollout paradigms: Constant-Action Rollout propagates the current action in an open-loop manner; Single-Prediction Rollout yields more plausible trajectories but remains open-loop; and Stepwise Prediction Rollout enables closed-loop, interaction-aware rollout.
• Figure 5: Representative scenarios of ForSim. The traffic agent (CBV) is marked in purple, AV (PDM-Lite Beibwenger2024pdmLite) is in red, and other agents are in blue.