Spectral-Geometric Neural Fields for Pose-Free LiDAR View Synthesis

Abstract

Neural Radiance Fields (NeRF) have shown remarkable success in image novel view synthesis (NVS), inspiring extensions to LiDAR NVS. However, most methods heavily rely on accurate camera poses for scene reconstruction. The sparsity and textureless nature of LiDAR data also present distinct challenges, leading to geometric holes and discontinuous surfaces. To address these issues, we propose SG-NLF, a pose-free LiDAR NeRF framework that integrates spectral information with geometric consistency. Specifically, we design a hybrid representation based on spectral priors to reconstruct smooth geometry. For pose optimization, we construct a confidence-aware graph based on feature compatibility to achieve global alignment. In addition, an adversarial learning strategy is introduced to enforce cross-frame consistency, thereby enhancing reconstruction quality. Comprehensive experiments demonstrate the effectiveness of our framework, especially in challenging low-frequency scenarios. Compared to previous state-of-the-art methods, SG-NLF improves reconstruction quality and pose accuracy by over 35.8% and 68.8%. Our work can provide a novel perspective for LiDAR view synthesis.

Figures (6)

• Figure 1: (a) Reconstruction and synthesis quality. The X-axis shows Chamfer Distance (CD) for point cloud reconstruction on the low-frequency KITTI-360 dataset xue2024geonlfliao2022kitti. The Y-axis shows RMSE for range depth and intensity synthesis. Circle area represents inference time. Lower CD and RMSE mean better reconstruction and synthesis. Our framework outperforms prior work by large margins. (b) Pose accuracy. The axis represents Absolute Trajectory Error (ATE) for pose estimation. Our SG-NLF demonstrates higher accuracy compared to existing NeRF-based models.
• Figure 2: Geometric Inconsistency tao2024lidarhuang2023neuralxue2024geonlf. The geometric hole mask is generated by comparing rendered opacity with ground truth LiDAR measurements. White regions in rectangular boxes (first row) highlight areas where these methods fail to reconstruct geometry. Comparisons between second and third rows further show these holes lead to poor-quality synthesized views.
• Figure 3: Overview of the SG-NLF. Given multi-view LiDAR sequences $\{\mathcal{S}_i\}_{i=0}^{N}$, SG-NLF employs a hybrid representation that combines discrete geometric encoding with continuous spectral embedding for effective reconstruction. The hybrid features are utilized to construct a confidence-aware graph for global pose optimization. The optimized poses and hybrid features are fed into NeRF to synthesize novel views $\hat{\mathcal{S}}$. To further enhance cross-frame consistency, SG-NLF distinguishes between the real depth maps (obtained from ground truth point clouds $\mathcal{S}_{ij}$ that are transformed into cross-frame coordinates) and fake depth maps (obtained from synthesized and transformed $\hat{\mathcal{S}}_{ij}$).
• Figure 4: Qualitative comparisons for LiDAR range depth and intensity reconstruction. Both pose-dependent zheng2024lidar4dli2022pcgen and pose-free methods xue2024geonlf are compared. Regions with obvious differences are highlighted in the rectangular boxes and arrows.
• Figure 5: Visual comparisons for pose estimation. We transform input sequences into a unified scene using estimated global poses and color-code the composite scene by height. We compare with pose-free methods lin2021barfxue2024geonlftao2024lidar. Best view zoomed in on-screen for details.