Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SymDrive: Realistic and Controllable Driving Simulator via Symmetric Auto-regressive Online Restoration

Zhiyuan Liu, Daocheng Fu, Pinlong Cai, Lening Wang, Ying Liu, Yilong Ren, Botian Shi, Jianqiang Wang

TL;DR

Sym Drive tackles data scarcity in autonomous driving by delivering a unified diffusion-based framework that jointly enables high-quality novel-view rendering and realistic traffic editing. It introduces a ground-truth-guided symmetric dual-view restoration and an autoregressive strategy to propagate detail across views while maintaining geometric consistency. A training-free harmonization process treats vehicle insertion as context-aware inpainting to ensure lighting and shadow coherence. Experiments on the Waymo Open Dataset show state-of-the-art performance in novel-view enhancement and 3D vehicle insertion, with real-time rendering under typical traffic densities.

Abstract

High-fidelity and controllable 3D simulation is essential for addressing the long-tail data scarcity in Autonomous Driving (AD), yet existing methods struggle to simultaneously achieve photorealistic rendering and interactive traffic editing. Current approaches often falter in large-angle novel view synthesis and suffer from geometric or lighting artifacts during asset manipulation. To address these challenges, we propose SymDrive, a unified diffusion-based framework capable of joint high-quality rendering and scene editing. We introduce a Symmetric Auto-regressive Online Restoration paradigm, which constructs paired symmetric views to recover fine-grained details via a ground-truth-guided dual-view formulation and utilizes an auto-regressive strategy for consistent lateral view generation. Furthermore, we leverage this restoration capability to enable a training-free harmonization mechanism, treating vehicle insertion as context-aware inpainting to ensure seamless lighting and shadow consistency. Extensive experiments demonstrate that SymDrive achieves state-of-the-art performance in both novel-view enhancement and realistic 3D vehicle insertion.

SymDrive: Realistic and Controllable Driving Simulator via Symmetric Auto-regressive Online Restoration

TL;DR

Sym Drive tackles data scarcity in autonomous driving by delivering a unified diffusion-based framework that jointly enables high-quality novel-view rendering and realistic traffic editing. It introduces a ground-truth-guided symmetric dual-view restoration and an autoregressive strategy to propagate detail across views while maintaining geometric consistency. A training-free harmonization process treats vehicle insertion as context-aware inpainting to ensure lighting and shadow coherence. Experiments on the Waymo Open Dataset show state-of-the-art performance in novel-view enhancement and 3D vehicle insertion, with real-time rendering under typical traffic densities.

Abstract

High-fidelity and controllable 3D simulation is essential for addressing the long-tail data scarcity in Autonomous Driving (AD), yet existing methods struggle to simultaneously achieve photorealistic rendering and interactive traffic editing. Current approaches often falter in large-angle novel view synthesis and suffer from geometric or lighting artifacts during asset manipulation. To address these challenges, we propose SymDrive, a unified diffusion-based framework capable of joint high-quality rendering and scene editing. We introduce a Symmetric Auto-regressive Online Restoration paradigm, which constructs paired symmetric views to recover fine-grained details via a ground-truth-guided dual-view formulation and utilizes an auto-regressive strategy for consistent lateral view generation. Furthermore, we leverage this restoration capability to enable a training-free harmonization mechanism, treating vehicle insertion as context-aware inpainting to ensure seamless lighting and shadow consistency. Extensive experiments demonstrate that SymDrive achieves state-of-the-art performance in both novel-view enhancement and realistic 3D vehicle insertion.
Paper Structure (14 sections, 8 equations, 7 figures, 5 tables)

This paper contains 14 sections, 8 equations, 7 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Visual Rendering and Generation
  4. High-fidelity Closed-loop Simulation
  5. Methodology
  6. Preliminaries
  7. Diffusion training
  8. Diffusion inference and 3DGS refinement
  9. Vehicle insertion and simulation
  10. Experiments
  11. Experiment Setup
  12. Novel view rendering
  13. Vehicle insertion and simulation
  14. Conclusion

Figures (7)

  • Figure 1: Challenges of existing visual simulation for AD system. Enlarge the image to see details
  • Figure 2: Novel-view restoration pipeline overview. a) The Gaussian Splatting (GS) model is trained separately for foreground vehicles and the background scene using ground truth (GT) images. b) Symmetric GS-rendered images are generated centered around the GT, and these symmetric data are used to train the diffusion model. c) Denoised novel view images are progressively generated via an autoregressive iterative process, and these images are then used to fine-tune the GS model.
  • Figure 3: Qualitative comparison with ReconDreamer Ni2024ReconDreamerCW and ReconDreamer++ zhao2025recon.
  • Figure 4: Qualitative comparison with Street Gaussians yan2024street and StreetCrafter yan2024streetcrafter.
  • Figure 5: Qualitative results of vehicle insertion and harmonization.
  • ...and 2 more figures