TerraSeg: Self-Supervised Ground Segmentation for Any LiDAR

Abstract

LiDAR perception is fundamental to robotics, enabling machines to understand their environment in 3D. A crucial task for LiDAR-based scene understanding and navigation is ground segmentation. However, existing methods are either handcrafted for specific sensor configurations or rely on costly per-point manual labels, severely limiting their generalization and scalability. To overcome this, we introduce TerraSeg, the first self-supervised, domain-agnostic model for LiDAR ground segmentation. We train TerraSeg on OmniLiDAR, a unified large-scale dataset that aggregates and standardizes data from 12 major public benchmarks. Spanning almost 22 million raw scans across 15 distinct sensor models, OmniLiDAR provides unprecedented diversity for learning a highly generalizable ground model. To supervise training without human annotations, we propose PseudoLabeler, a novel module that generates high-quality ground and non-ground labels through self-supervised per-scan runtime optimization. Extensive evaluations demonstrate that, despite using no manual labels, TerraSeg achieves state-of-the-art results on nuScenes, SemanticKITTI, and Waymo Perception while delivering real-time performance. Our code and model weights are publicly available.

Figures (9)

• Figure 1: Ground segmentation performance across diverse LiDAR benchmarks for baseline PatchWork++ lee2022patchwork++ and our TerraSeg. We present a qualitative comparison between TerraSeg and the baseline on the nuScenes caesar2020nuscenes, SemanticKITTI behley2019semantickitti, and Waymo Perception sun2020scalability datasets. Ground points are colored based on elevation, non-ground points are gray, and highlighted regions are marked with red.
• Figure 2: Overview of TerraSeg. (a) PseudoLabeler generates self-supervised point-wise ground/non-ground labels per raw LiDAR scan. (b) OmniLiDAR unifies 12 major automotive datasets within an aggregated framework, yielding a diverse curated corpus drawn from nearly 22 million raw scans. (c) TerraSeg is a real-time, domain-agnostic model for ground segmentation, trained on OmniLiDAR using these self-supervised pseudo-labels. We introduce two architectures: accurate Base model (TerraSeg-B) and efficient Small model (TerraSeg-S).
• Figure 3: Post-processing for pseudo-labeling. The point cloud is divided into pillars to recover misclassified non-ground points. Points within a vertical window defined by $[ \hat{h} - H_{p1}, \, \hat{h} + H_{p2}]$ around the estimated height $\hat{h} = g (x_i, y_i)$ are analyzed: if a pillar contains both ground and non-ground points, all points above the pillar’s lowest point plus margin $\tau$ are reclassified as non-ground.
• Figure 4: Qualitative success cases on nuScenes dataset caesar2020nuscenes. The top row shows ground truth labels, while the bottom row displays TerraSeg predictions. Ground points are color-coded by elevation; non-ground points are rendered in gray. From left to right, columns correspond to scan IDs 1787.0, 3238.0, and 3479.0.
• Figure 5: Qualitative success cases on SemanticKITTI dataset behley2019semantickitti. The top row shows ground truth labels, while the bottom row displays TerraSeg predictions. Ground points are color-coded by elevation; non-ground points are rendered in gray. From left to right, columns correspond to scan IDs 4006.0, 2714.0, and 2816.0.