Subspace Prototype Guidance for Mitigating Class Imbalance in Point Cloud Semantic Segmentation

TL;DR

This paper introduces a novel method, namely subspace prototype guidance (SPG), to guide the training of segmentation network, which significantly improves the segmentation performance and surpasses the state-of-the-art method.

Abstract

Point cloud semantic segmentation can significantly enhance the perception of an intelligent agent. Nevertheless, the discriminative capability of the segmentation network is influenced by the quantity of samples available for different categories. To mitigate the cognitive bias induced by class imbalance, this paper introduces a novel method, namely subspace prototype guidance (\textbf{SPG}), to guide the training of segmentation network. Specifically, the point cloud is initially separated into independent point sets by category to provide initial conditions for the generation of feature subspaces. The auxiliary branch which consists of an encoder and a projection head maps these point sets into separate feature subspaces. Subsequently, the feature prototypes which are extracted from the current separate subspaces and then combined with prototypes of historical subspaces guide the feature space of main branch to enhance the discriminability of features of minority categories. The prototypes derived from the feature space of main branch are also employed to guide the training of the auxiliary branch, forming a supervisory loop to maintain consistent convergence of the entire network. The experiments conducted on the large public benchmarks (i.e. S3DIS, ScanNet v2, ScanNet200, Toronto-3D) and collected real-world data illustrate that the proposed method significantly improves the segmentation performance and surpasses the state-of-the-art method. The code is available at \url{https://github.com/Javion11/PointLiBR.git}.

Figures (12)

• Figure 1: Semantic understanding of SPG. The input point cloud is abstracted into high-dimensional features through a representation network. Then these features are assigned the corresponding category labels, which belong to the closest prototypes in the metric space.
• Figure 2: The general process of point cloud semantic segmentation networks (including PointNet++qi2017pointnet++, PTv1zhao2021point, PTv2wu2022point, RandLANethu2020randla, KPConvthomas2019kpconv, etc.).
• Figure 3: The values of feature centers of "ceiling" and "window" in each dimension.
• Figure 4: The feature distributions of "ceiling" and "window". They are visualized by t-SNEvan2008visualizing.
• Figure 5: The overall framework of SPG. The grouped point sets are projected into separate feature subspaces via a plug-and-play auxiliary branch, facilitating the derivation of discriminative prototypes. The feature prototypes of the current scene, in combination with those from historical scenes, provide supervision for the feature space of the main branch to reduce the intra-class variance. Simultaneously, the features correctly classified by the main branch also supervise the feature space of the auxiliary branch, ensuring their convergence aligning with each other.