Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LiDAR-EVS: Enhance Extrapolated View Synthesis for 3D Gaussian Splatting with Pseudo-LiDAR Supervision

Yiming Huang, Xin Kang, Sipeng Zhang, Hongliang Ren, Weihua Zhang, Junjie Lai

Abstract

3D Gaussian Splatting (3DGS) has emerged as a powerful technique for real-time LiDAR and camera synthesis in autonomous driving simulation. However, simulating LiDAR with 3DGS remains challenging for extrapolated views beyond the training trajectory, as existing methods are typically trained on single-traversal sensor scans, suffer from severe overfitting and poor generalization to novel ego-vehicle paths. To enable reliable simulation of LiDAR along unseen driving trajectories without external multi-pass data, we present LiDAR-EVS, a lightweight framework for robust extrapolated-view LiDAR simulation in autonomous driving. Designed to be plug-and-play, LiDAR-EVS readily extends to diverse LiDAR sensors and neural rendering baselines with minimal modification. Our framework comprises two key components: (1) pseudo extrapolated-view point cloud supervision with multi-frame LiDAR fusion, view transformation, occlusion curling, and intensity adjustment; (2) spatially-constrained dropout regularization that promotes robustness to diverse trajectory variations encountered in real-world driving. Extensive experiments demonstrate that LiDAR-EVS achieves SOTA performance on extrapolated-view LiDAR synthesis across three datasets, making it a promising tool for data-driven simulation, closed-loop evaluation, and synthetic data generation in autonomous driving systems.

LiDAR-EVS: Enhance Extrapolated View Synthesis for 3D Gaussian Splatting with Pseudo-LiDAR Supervision

Abstract

3D Gaussian Splatting (3DGS) has emerged as a powerful technique for real-time LiDAR and camera synthesis in autonomous driving simulation. However, simulating LiDAR with 3DGS remains challenging for extrapolated views beyond the training trajectory, as existing methods are typically trained on single-traversal sensor scans, suffer from severe overfitting and poor generalization to novel ego-vehicle paths. To enable reliable simulation of LiDAR along unseen driving trajectories without external multi-pass data, we present LiDAR-EVS, a lightweight framework for robust extrapolated-view LiDAR simulation in autonomous driving. Designed to be plug-and-play, LiDAR-EVS readily extends to diverse LiDAR sensors and neural rendering baselines with minimal modification. Our framework comprises two key components: (1) pseudo extrapolated-view point cloud supervision with multi-frame LiDAR fusion, view transformation, occlusion curling, and intensity adjustment; (2) spatially-constrained dropout regularization that promotes robustness to diverse trajectory variations encountered in real-world driving. Extensive experiments demonstrate that LiDAR-EVS achieves SOTA performance on extrapolated-view LiDAR synthesis across three datasets, making it a promising tool for data-driven simulation, closed-loop evaluation, and synthetic data generation in autonomous driving systems.
Paper Structure (16 sections, 7 equations, 8 figures, 10 tables, 2 algorithms)

This paper contains 16 sections, 7 equations, 8 figures, 10 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Related Works
  3. Preliminary
  4. LiDAR-EVS
  5. Sensor Agnostic Pseudo LiDAR Curation
  6. Spatially-Constrained Dropout Regularization
  7. Training Strategy and Optimization
  8. Experiments
  9. Settings
  10. Result Analysis
  11. Ablation Study
  12. Conclusion
  13. Additional Implementation Details
  14. Computational Overhead
  15. Generalization to other Baselines
  16. ...and 1 more sections

Figures (8)

  • Figure 1: LiDAR-EVS enables robust LiDAR synthesis for extrapolated views beyond the training trajectory. Given single-traversal data, our framework generates pseudo supervision for novel viewpoints, achieving accurate LiDAR simulation on extrapolation view, outperforming existing methods that overfit to the training path.
  • Figure 2: LiDAR-EVS Pipeline. Our framework consists of two key modules: Pseudo LiDAR Curation and Spatially Constrained Dropout. Pseudo LiDAR curation include the following steps: (1) Multi-frame fusion, (2) Extrapolated view transformation, (3) Occlusion curling, (4) Intensity adjustment. With the proposed framework, we can optimize the Gaussian scene representation to achieve robust LiDAR synthesis for both interpolated and extrapolated view rendering.
  • Figure 3: Qualitative LiDAR Rendering Results on Para-Lane Dataset.
  • Figure 4: Qualitative LiDAR Rendering Results on nuScenes and Pandaset.
  • Figure 5: Qualitative Image Rendering Results on nuScenes, Pandaset and Para-Lane Dataset.
  • ...and 3 more figures