LiSu: A Dataset and Method for LiDAR Surface Normal Estimation

TL;DR

This work addresses the lack of large-scale LiDAR surface normal annotations by introducing LiSu, a CARLA-based synthetic dataset with ground-truth normals. It presents a one-shot LiDAR normal estimator built on Point Transformer V3, augmented with spatial and temporal Graph Total Variation regularizers and an Eikonal norm constraint to enforce coherence and unit-norm behavior. The approach achieves state-of-the-art results on LiSu, demonstrates strong synthetic-to-real transfer to Waymo data, and improves neural surface reconstruction when normals are incorporated. Overall, LiSu and the regularized training framework provide a scalable, deployment-friendly path for LiDAR-based surface normal estimation in autonomous driving contexts.

Abstract

While surface normals are widely used to analyse 3D scene geometry, surface normal estimation from LiDAR point clouds remains severely underexplored. This is caused by the lack of large-scale annotated datasets on the one hand, and lack of methods that can robustly handle the sparse and often noisy LiDAR data in a reasonable time on the other hand. We address these limitations using a traffic simulation engine and present LiSu, the first large-scale, synthetic LiDAR point cloud dataset with ground truth surface normal annotations, eliminating the need for tedious manual labeling. Additionally, we propose a novel method that exploits the spatiotemporal characteristics of autonomous driving data to enhance surface normal estimation accuracy. By incorporating two regularization terms, we enforce spatial consistency among neighboring points and temporal smoothness across consecutive LiDAR frames. These regularizers are particularly effective in self-training settings, where they mitigate the impact of noisy pseudo-labels, enabling robust real-world deployment. We demonstrate the effectiveness of our method on LiSu, achieving state-of-the-art performance in LiDAR surface normal estimation. Moreover, we showcase its full potential in addressing the challenging task of synthetic-to-real domain adaptation, leading to improved neural surface reconstruction on real-world data.

Figures (14)

• Figure 1: Our synthetic LiSu dataset \ref{['fig:teaser_lisu']} enables focusing research on the challenging task of LiDAR surface normal estimation. When combined with our proposed method, we achieve state-of-the-art results \ref{['fig:teaser_lisu']} on challenging real-world datasets like Waymo Open Dataset sunScalabilityPerceptionAutonomous2020, outperforming the current state-of-the-art SHS-Net liSHSNetLearningSigned2023\ref{['fig:teaser_lisu']}. Best viewed in color on screen.
• Figure 2: Spherical KDE plot using a von Mises-Fisher kernel to visualize surface normal distribution. Yellow regions indicate higher density of surface normals. White regions in the south correspond to physically impossible orientations, while those in the north represent extremely rare occurrences.
• Figure 3: Exemplary LiDAR frames from our LiSu dataset (left: tunnel portal, right: urban scene). Surface normals are linearly mapped to the RGB color space. The color legend spheres in the bottom right corners provide a visual reference to interpret the normal directions.
• Figure 4: Qualitative evaluation of traditional and learning-based methods on a challenging Waymo frame. Notably, the current state-of-the-art (SHS-Net liSHSNetLearningSigned2023 trained on PCPNet guerreroPCPNetLearningLocal2018) struggles to generalize to noisy and sparse LiDAR data. When trained on our proposed dataset (LiSu), both SHS-Net and our method yield reasonable results on the Waymo frame. In the last row, we demonstrate how leveraging our method within a self-supervised learning paradigm significantly enhances overall estimation, resulting in accurate, smooth, and consistently oriented predictions.
• Figure 5: Reconstructed meshes from Waymo sequence 1172406, color-coded with surface normals. ReSimAD zhangReSimADZeroShot3D2024 omits surface normal loss during reconstruction. Direct Transfer (DT) applies a model trained on LiSu without adaptation. $\mathbf{\mathcal{L}}$ (\ref{['eq:total_loss']}) and $\mathbf{\mathcal{L}_{L1}}$ (\ref{['eq:loss_l1']}) denote Waymo self-trained models with and without regularization, respectively.