Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ViSE: A Systematic Approach to Vision-Only Street-View Extrapolation

Kaiyuan Tan, Yingying Shen, Haiyang Sun, Bing Wang, Guang Chen, Hangjun Ye

TL;DR

This work tackles the challenge of extrapolating realistic street views for closed-loop autonomous driving simulations. It introduces a four-stage, LiDAR-free pipeline that combines vision-based pseudo-LiDAR initialization, a dimension-reduced 2D-SDF road prior, iterative pseudo-ground-truth supervision via a diffusion-based fixer, and a Time-Invariance Adaptation Network to remove time-specific artifacts. The approach yields state-of-the-art performance on RealADSim-NVS, achieving a final score of $0.441$ and first place, while ablations demonstrate the additive value of each component, with notable gains from the 2D-SDF road prior, pseudo-GT supervision, and temporal regularization. Overall, the method delivers geometrically consistent extrapolations and temporally robust results, enabling more realistic and reliable closed-loop autonomous driving simulations.

Abstract

Realistic view extrapolation is critical for closed-loop simulation in autonomous driving, yet it remains a significant challenge for current Novel View Synthesis (NVS) methods, which often produce distorted and inconsistent images beyond the original trajectory. This report presents our winning solution which ctook first place in the RealADSim Workshop NVS track at ICCV 2025. To address the core challenges of street view extrapolation, we introduce a comprehensive four-stage pipeline. First, we employ a data-driven initialization strategy to generate a robust pseudo-LiDAR point cloud, avoiding local minima. Second, we inject strong geometric priors by modeling the road surface with a novel dimension-reduced SDF termed 2D-SDF. Third, we leverage a generative prior to create pseudo ground truth for extrapolated viewpoints, providing auxilary supervision. Finally, a data-driven adaptation network removes time-specific artifacts. On the RealADSim-NVS benchmark, our method achieves a final score of 0.441, ranking first among all participants.

ViSE: A Systematic Approach to Vision-Only Street-View Extrapolation

TL;DR

This work tackles the challenge of extrapolating realistic street views for closed-loop autonomous driving simulations. It introduces a four-stage, LiDAR-free pipeline that combines vision-based pseudo-LiDAR initialization, a dimension-reduced 2D-SDF road prior, iterative pseudo-ground-truth supervision via a diffusion-based fixer, and a Time-Invariance Adaptation Network to remove time-specific artifacts. The approach yields state-of-the-art performance on RealADSim-NVS, achieving a final score of and first place, while ablations demonstrate the additive value of each component, with notable gains from the 2D-SDF road prior, pseudo-GT supervision, and temporal regularization. Overall, the method delivers geometrically consistent extrapolations and temporally robust results, enabling more realistic and reliable closed-loop autonomous driving simulations.

Abstract

Realistic view extrapolation is critical for closed-loop simulation in autonomous driving, yet it remains a significant challenge for current Novel View Synthesis (NVS) methods, which often produce distorted and inconsistent images beyond the original trajectory. This report presents our winning solution which ctook first place in the RealADSim Workshop NVS track at ICCV 2025. To address the core challenges of street view extrapolation, we introduce a comprehensive four-stage pipeline. First, we employ a data-driven initialization strategy to generate a robust pseudo-LiDAR point cloud, avoiding local minima. Second, we inject strong geometric priors by modeling the road surface with a novel dimension-reduced SDF termed 2D-SDF. Third, we leverage a generative prior to create pseudo ground truth for extrapolated viewpoints, providing auxilary supervision. Finally, a data-driven adaptation network removes time-specific artifacts. On the RealADSim-NVS benchmark, our method achieves a final score of 0.441, ranking first among all participants.
Paper Structure (16 sections, 8 equations, 1 figure, 2 tables)

This paper contains 16 sections, 8 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. 3D Gaussian Splatting
  4. Rendering Signed Distance Functions
  5. Method
  6. Problem Formulation
  7. 3D Scene Initialization without LiDAR
  8. Geometry-Aware 3D Scene Reconstruction
  9. 2D-SDF for Road Surface Representation
  10. Sky Modeling and Composition
  11. Iterative Pseudo Ground Truth Generation
  12. Time-Invariance Adaptation Network
  13. Experiments
  14. Main Results
  15. Ablation Study
  16. ...and 1 more sections

Figures (1)

  • Figure 1: Proposed Pipeline. We introduce a four-stage framework for consistent novel-view synthesis. The process begins with (1) generating a robust, vision-based point cloud for initialization. This enables (2) detailed 3D reconstruction, where a 2D-SDF provides strong geometric priors for the road. To address remaining artifacts in unobserved areas, we employ (3) an iterative pseudo-GT strategy using a generative prior. Finally, (4) a time-invariance adaptation network ensures the output is temporally invariant across different driving logs.