STONE Dataset: A Scalable Multi-Modal Surround-View 3D Traversability Dataset for Off-Road Robot Navigation

TL;DR

STONE provides trajectory-guided 3D traversability maps generated by a fully automated, annotation-free pipeline, and comprehensive surround-view sensing with synchronized 128-channel LiDAR, six RGB cameras, and three 4D imaging radars for off-road navigation.

Abstract

Reliable off-road navigation requires accurate estimation of traversable regions and robust perception under diverse terrain and sensing conditions. However, existing datasets lack both scalability and multi-modality, which limits progress in 3D traversability prediction. In this work, we introduce STONE, a large-scale multi-modal dataset for off-road navigation. STONE provides (1) trajectory-guided 3D traversability maps generated by a fully automated, annotation-free pipeline, and (2) comprehensive surround-view sensing with synchronized 128-channel LiDAR, six RGB cameras, and three 4D imaging radars. The dataset covers a wide range of environments and conditions, including day and night, grasslands, farmlands, construction sites, and lakes. Our auto-labeling pipeline reconstructs dense terrain surfaces from LiDAR scans, extracts geometric attributes such as slope, elevation, and roughness, and assigns traversability labels beyond the robot's trajectory using a Mahalanobis-distance-based criterion. This design enables scalable, geometry-aware ground-truth construction without manual annotation. Finally, we establish a benchmark for voxel-level 3D traversability prediction and provide strong baselines under both single-modal and multi-modal settings. STONE is available at: https://konyul.github.io/STONE-dataset/

Figures (7)

• Figure 1: Overview of the STONE dataset. The STONE dataset is a multi-modal 3D traversability dataset collected in off-road environments, which provides ground-truth annotations of 3D traversable areas automatically without human effort. (a) illustrates surround-view images captured around the robot. (b) shows the 3D scene captured by the multi-modal surround-view obtained using LiDAR, cameras, and 4D radar. (c) presents the process of automatically generating scalable 3D traversability maps based on robot trajectories.
• Figure 2: Sensor setup and coverage of the Bunker Pro UGV platform. (a) shows the sensor placement including LiDAR, cameras, 4D radars, IMU and GPS. (b) illustrates the sensing range and coverage in off-road environments.
• Figure 3: Various conditions in which the dataset was collected. (a) illustrates off-road terrains such as sandy, rocky, gravelly, puddled, and muddy. (b) shows diverse illumination conditions in the dataset: sunny, cloudy, shadow, sunset, and night.
• Figure 4: Dataset composition across different environments.
• Figure 5: Overview of automated 3D traversability map generation process. (a) LiDAR point clouds are accumulated over multiple time steps in the global coordinate system and reconstructed into a 3D mesh using Poisson surface reconstruction. (b) For each vertex of the 3D mesh, geometric features such as elevation, slope, and roughness are extracted to construct a traversability map. (c) A reference distribution is computed from the geometric features of voxels along the robot’s driving trajectories using a multivariate Gaussian, and a 3D traversability ground-truth label is automatically generated for each voxel whose Mahalanobis distance falls below a predefined threshold.