DRINet++: Efficient Voxel-as-point Point Cloud Segmentation

TL;DR

DRINet++ addresses the memory–efficiency–performance tradeoff in outdoor LiDAR segmentation by treating voxels as points and operating on a single sparse voxel representation. It introduces Sparse Feature Encoder and Sparse Geometry Feature Enhancement, with Multi-scale Sparse Projection and Attentive Multi-scale Fusion, and applies Deep Sparse Supervision to accelerate training with minimal memory overhead. Experiments on SemanticKITTI and Nuscenes demonstrate state-of-the-art performance and real-time inference on standard GPUs, with ablations confirming the contribution of each component and the memory benefits of the voxel‑as‑point design. The work offers a practical, scalable approach for high-accuracy outdoor 3D segmentation suitable for real-world deployment.

Abstract

Recently, many approaches have been proposed through single or multiple representations to improve the performance of point cloud semantic segmentation. However, these works do not maintain a good balance among performance, efficiency, and memory consumption. To address these issues, we propose DRINet++ that extends DRINet by enhancing the sparsity and geometric properties of a point cloud with a voxel-as-point principle. To improve efficiency and performance, DRINet++ mainly consists of two modules: Sparse Feature Encoder and Sparse Geometry Feature Enhancement. The Sparse Feature Encoder extracts the local context information for each point, and the Sparse Geometry Feature Enhancement enhances the geometric properties of a sparse point cloud via multi-scale sparse projection and attentive multi-scale fusion. In addition, we propose deep sparse supervision in the training phase to help convergence and alleviate the memory consumption problem. Our DRINet++ achieves state-of-the-art outdoor point cloud segmentation on both SemanticKITTI and Nuscenes datasets while running significantly faster and consuming less memory.

Figures (5)

• Figure 1: Comparison between two common ways to deal with sparse features and our proposed method.
• Figure 2: The overall structure of our DRINet++. In the top half of the figure, LiDAR input is firstly voxelized as sparse features. Then the Sparse Feature Encoder utilizes sparse convolutions to process the sparse features. Furthermore, Sparse Geometry Feature Enhancement will enhance the features by Multi-scale Sparse Projection and Attentive Multi-scale Fusion layer to generate the input of Sparse Feature Encoder at the next stage. Sparse supervision will be attached to the output of the sparse feature encoder as an auxiliary loss. The bottom line describes the details about Multi-scale Sparse Projection and Attentive Multi-scale Fusion. $N$ is the number of points, $M_{i}$ is the number of voxels for $i$-th scale, $C_E$ is the channel dimension.
• Figure 3: The variation curve of the ratio of voxel number ($N_{voxels}$) to point number ($N_{points}$) against voxel size.
• Figure 4: An example demonstrates the Attentive Multi-scale Fusion. $\otimes$ means tensor elementwise multiplication.
• Figure 5: The results on the SemanticKITTI valid set. The top row is the ground truth, and the bottom row is the predictions by DRINet++.