LiDAR-GS:Real-time LiDAR Re-Simulation using Gaussian Splatting

TL;DR

LiDAR-GS presents a real-time LiDAR re-simulation framework based on Gaussian Splatting that embraces range-view organization, differentiable laser beam splatting via Micro Cross-Section Projection, and Neural Gaussian Representation to model view- and distance-dependent LiDAR properties. By decomposing dynamic instances and applying learnable ray-drop alongside explicit AABB-based projection constraints, the approach achieves high-fidelity depth and intensity reconstruction while maintaining real-time performance on public driving datasets. The combination of these components yields state-of-the-art results versus explicit mesh and NeRF-based LiDAR re-simulation methods, with strong generalization to novel views and dynamic scenes. This work advances data closure and evaluation for autonomous driving by enabling scalable, fast, and accurate LiDAR re-simulation, and it provides publicly available code for broader adoption.

Abstract

We present LiDAR-GS, a Gaussian Splatting (GS) method for real-time, high-fidelity re-simulation of LiDAR scans in public urban road scenes. Recent GS methods proposed for cameras have achieved significant advancements in real-time rendering beyond Neural Radiance Fields (NeRF). However, applying GS representation to LiDAR, an active 3D sensor type, poses several challenges that must be addressed to preserve high accuracy and unique characteristics. Specifically, LiDAR-GS designs a differentiable laser beam splatting, using range-view representation for precise surface splatting by projecting lasers onto micro cross-sections, effectively eliminating artifacts associated with local affine approximations. Furthermore, LiDAR-GS leverages Neural Gaussian Representation, which further integrate view-dependent clues, to represent key LiDAR properties that are influenced by the incident direction and external factors. Combining these practices with some essential adaptations, e.g., dynamic instances decomposition, LiDAR-GS succeeds in simultaneously re-simulating depth, intensity, and ray-drop channels, achieving state-of-the-art results in both rendering frame rate and quality on publically available large scene datasets when compared with the methods using explicit mesh or implicit NeRF. Our source code is publicly available at https://www.github.com/cqf7419/LiDAR-GS.

Figures (10)

• Figure 1: We present LiDAR-GS, the LiDAR Gaussian Splatting for re-simulating LiDAR frames rendered in real-time.
• Figure 2: The overview of LiDAR-GS pipeline. We choose to represent LiDAR frames as range-view images (\ref{['sec:method:pred:rangeview']}), and use Gaussian Splatting to perform both reconstruction and rendering (\ref{['sec:method:pred:gs']}). Specifically, we choose to use Neural Gaussian Representation (NGR) with four MLPs to learn final attributes for each Gaussian (\ref{['sec:method:lidar-gs']}), where projecting and rasterizing these LiDAR Gaussians are performed through our proposed Laser Beam Splatting strategy (\ref{['sec:method:laser-beam-splatting']}). Dynamic instances are separately reconstructed in their stand-alone coordinates for final stitching (\ref{['sec:method:dyn']}).
• Figure 3: Illustration of the proposed Micro Cross-Section Projection. Instead of splatting Gaussians to the normalized image plane like 3DGS Kerbl20233dgs, we calculate the cross-section micro-plane through the ray established between Gaussian mean and sensor viewpoint for splatting. During forward rendering, we collect acrossed Gaussians along the ray direction to integrate attributes for each pixel.
• Figure 4: Sideview of the back-projection for each Gaussian during rendering.
• Figure 5: Dynamic object decomposition and Gaussian modeling. Given detection and segmentation results of each moving instance between consecutive frames, we separately stitch and reconstruct them in a standalone coordinate system for improving the completeness.