Filling Missing Values Matters for Range Image-Based Point Cloud Segmentation

TL;DR

This work tackles the detrimental impact of missing values in range-image representations for LiDAR PCS by introducing Scan Unfolding++ (SU++) to reduce missing data, and a simple yet effective range-dependent $K$-nearest neighbor interpolation ($K$NNI) to fill residual gaps. Building on these inputs, the Filling Missing Values Network (FMVNet) and its Fast variant achieve state-of-the-art speed-accuracy trade-offs for range-image based segmentation, demonstrated across SemanticKITTI, SemanticPOSS, and nuScenes. The authors provide rigorous evaluation of projection strategies, interpolation techniques, and architectural choices, establishing SU++ and $K$NNI as broadly beneficial for range-image tasks. The methods offer practical impact for robot perception and navigation, with potential extension to moving-object segmentation and multimodal systems.

Abstract

Point cloud segmentation (PCS) plays an essential role in robot perception and navigation tasks. To efficiently understand large-scale outdoor point clouds, their range image representation is commonly adopted. This image-like representation is compact and structured, making range image-based PCS models practical. However, undesirable missing values in the range images damage the shapes and patterns of objects. This problem creates difficulty for the models in learning coherent and complete geometric information from the objects. Consequently, the PCS models only achieve inferior performance. Delving deeply into this issue, we find that the use of unreasonable projection approaches and deskewing scans mainly leads to unwanted missing values in the range images. Besides, almost all previous works fail to consider filling in the unexpected missing values in the PCS task. To alleviate this problem, we first propose a new projection method, namely scan unfolding++ (SU++), to avoid massive missing values in the generated range images. Then, we introduce a simple yet effective approach, namely range-dependent $K$-nearest neighbor interpolation ($K$NNI), to further fill in missing values. Finally, we introduce the Filling Missing Values Network (FMVNet) and Fast FMVNet. Extensive experimental results on SemanticKITTI, SemanticPOSS, and nuScenes datasets demonstrate that by employing the proposed SU++ and $K$NNI, existing range image-based PCS models consistently achieve better performance than the baseline models. Besides, both FMVNet and Fast FMVNet achieve state-of-the-art performance in terms of the speed-accuracy trade-off. The proposed methods can be applied to other range image-based tasks and practical applications.

Figures (10)

• Figure 1: (a) The image produced by spherical projection rangenet++ on the deskewing scan in the SemanticKITTI semantickitti_2019_behley dataset. There are many missing values. Specifically, lasers not evenly spaced in the vertical direction lead to black horizontal lines (emphasized by the white rectangle mask). Besides, deskewing scans (after motion compensation) cause large missing values (highlighted by the white ellipse mask). (b) The image generated by the proposed scan unfolding++ on the deskewing scan. All black horizontal lines (missing values) have been removed. (c) The image made by scan unfolding++ on the skewing scan. The large missing values within the white ellipse mask have been filled in. (d) The image produced by scan unfolding++ on the raw scan. It is used for comparison with the image in (c). (e) The image after applying the proposed range-dependent $K$-nearest neighbor interpolation ($K$NNI) on the image in (c). Many missing values (small black points) have been filled in valid values. All objects such as the bicyclist, car, and road appear coherent and complete.
• Figure 2: The range images are first generated by the projection method, namely scan unfolding++. Then, we apply range-dependent $K$-nearest neighbor interpolation ($K$NNI) on the images to fill in partial missing points. Subsequently, the images go through the range image-based network, namely FMVNet, to predict the labels. Finally, the outputs are projected back onto the points and pass through the post-processing approach (Nearest Label Assignment fidnet_2021) to obtain the final predictions.
• Figure 3: SemanticKITTI semantickitti_2019_behley data representation in a scan. Lasers $1\sim64$ generate $n_1, n_2, \dots, n_{64}$ 3D points, respectively. The azimuth degree gap between the last point (see $\boldsymbol{p}_{1,n_1}$) in the current scan line and the first point in the next scan line (see $\boldsymbol{p}_{2,1}$) is $360^{\circ}$ theoretically.
• Figure 4: Look-up table for projecting points onto the range image. The first row stores the indices of the points in a scan. The second and third rows store the corresponding $\boldsymbol{v}$ and $\boldsymbol{u}$ coordinates in the range image.
• Figure 5: Projection of points onto the range image.