Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hybrid Gaussian Splatting for Novel Urban View Synthesis

Mohamed Omran, Farhad Zanjani, Davide Abati, Jens Petersen, Amirhossein Habibian

TL;DR

The paper tackles urban novel-view synthesis (NVS) by proposing a hybrid pipeline that combines explicit 3D reconstruction with learned frame refinement. A 3D Gaussian Splatting (3DGS) backend reconstructs the scene geometry and bootstraps novel views, which are then polished by a diffusion-based enhancer to remove rendering artifacts. To train the enhancer, the authors curate a sizable paired dataset from Paralane and EUVS, mask dynamic objects during training, and optimize a weighted loss with $L1$, SSIM, and LPIPS, achieving robust improvements when using higher-resolution inputs and better point-cloud initialization. The approach achieves second place on the RealADSim-NVS leaderboard with an aggregated score of $0.432$, demonstrating the viability of combining explicit 3D representations with diffusion-based refinement for challenging autonomous-driving NVS tasks.

Abstract

This paper describes the Qualcomm AI Research solution to the RealADSim-NVS challenge, hosted at the RealADSim Workshop at ICCV 2025. The challenge concerns novel view synthesis in street scenes, and participants are required to generate, starting from car-centric frames captured during some training traversals, renders of the same urban environment as viewed from a different traversal (e.g. different street lane or car direction). Our solution is inspired by hybrid methods in scene generation and generative simulators merging gaussian splatting and diffusion models, and it is composed of two stages: First, we fit a 3D reconstruction of the scene and render novel views as seen from the target cameras. Then, we enhance the resulting frames with a dedicated single-step diffusion model. We discuss specific choices made in the initialization of gaussian primitives as well as the finetuning of the enhancer model and its training data curation. We report the performance of our model design and we ablate its components in terms of novel view quality as measured by PSNR, SSIM and LPIPS. On the public leaderboard reporting test results, our proposal reaches an aggregated score of 0.432, achieving the second place overall.

Hybrid Gaussian Splatting for Novel Urban View Synthesis

TL;DR

The paper tackles urban novel-view synthesis (NVS) by proposing a hybrid pipeline that combines explicit 3D reconstruction with learned frame refinement. A 3D Gaussian Splatting (3DGS) backend reconstructs the scene geometry and bootstraps novel views, which are then polished by a diffusion-based enhancer to remove rendering artifacts. To train the enhancer, the authors curate a sizable paired dataset from Paralane and EUVS, mask dynamic objects during training, and optimize a weighted loss with , SSIM, and LPIPS, achieving robust improvements when using higher-resolution inputs and better point-cloud initialization. The approach achieves second place on the RealADSim-NVS leaderboard with an aggregated score of , demonstrating the viability of combining explicit 3D representations with diffusion-based refinement for challenging autonomous-driving NVS tasks.

Abstract

This paper describes the Qualcomm AI Research solution to the RealADSim-NVS challenge, hosted at the RealADSim Workshop at ICCV 2025. The challenge concerns novel view synthesis in street scenes, and participants are required to generate, starting from car-centric frames captured during some training traversals, renders of the same urban environment as viewed from a different traversal (e.g. different street lane or car direction). Our solution is inspired by hybrid methods in scene generation and generative simulators merging gaussian splatting and diffusion models, and it is composed of two stages: First, we fit a 3D reconstruction of the scene and render novel views as seen from the target cameras. Then, we enhance the resulting frames with a dedicated single-step diffusion model. We discuss specific choices made in the initialization of gaussian primitives as well as the finetuning of the enhancer model and its training data curation. We report the performance of our model design and we ablate its components in terms of novel view quality as measured by PSNR, SSIM and LPIPS. On the public leaderboard reporting test results, our proposal reaches an aggregated score of 0.432, achieving the second place overall.
Paper Structure (14 sections, 4 equations, 4 figures, 1 table)

This paper contains 14 sections, 4 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. NVS Problem formulation
  3. Hybrid Gaussian Splatting for NVS
  4. Gaussian Splatting Backend
  5. Diffusion Enhancement
  6. Training data curation.
  7. Training loss.
  8. Training resolution.
  9. Iterative 3DGS reconstruction and rendering.
  10. Results
  11. Implementation and evaluation details.
  12. 3DGS improvements.
  13. Enhancer results.
  14. Qualitative results.

Figures (4)

  • Figure 1: Our proposed hybrid NVS pipeline. First, (a) the source frames and the corresponding views are used to fit a 3DGS model of the scene (blue arrows), that can be used to render frames at the desired target view (green arrows). Then, (b) the rendered view from 3DGS is refined by a dedicated enhancer model, along with a reference source frame (gray arrows). Optionally, the enhanced render can be used as a pseudo source view to further refine the 3DGS reconstruction (red arrow).
  • Figure 2: Qualitative results of our approach. Fist three rows: our 3DGS backend bootstraps the novel view, creating a frame from the correct viewpoint but featuring novel-view artifacts, especially close to the camera. The enhancer model then polishes the frame yielding the final frame. Last row: a failure case of our approach. Whenever the distance between source and target viewpoints is too high, the 3DGS backend fails completely, and the enhancer cannot recover a meaningful frame.
  • Figure 3: Impact of point cloud initialization on the 3DGS performance. By initializing the gaussian locations with more points, the overall NVS performance improves. Numbers reported on the challenge sequences, in the format 'source views / target views'.
  • Figure 4: Visual results of iterative 3DGS refinement. Refining the 3DGS representation with $K=3$ refinement steps improves the final quality of the novel view render.