Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Hierarchical Insights: Exploiting Structural Similarities for Reliable 3D Semantic Segmentation

Mariella Dreissig, Simon Ruehle, Florian Piewak, Joschka Boedecker

TL;DR

This work proposes a training strategy for a 3D LiDAR semantic segmentation model that learns structural relationships between classes through abstraction by implicitly modeling these relationships using a learning rule for hierarchical multi-label classification (HMC).

Abstract

Safety-critical applications such as autonomous driving require robust 3D environment perception algorithms capable of handling diverse and ambiguous surroundings. The predictive performance of classification models is heavily influenced by the dataset and the prior knowledge provided by the annotated labels. While labels guide the learning process, they often fail to capture the inherent relationships between classes that are naturally understood by humans. We propose a training strategy for a 3D LiDAR semantic segmentation model that learns structural relationships between classes through abstraction. This is achieved by implicitly modeling these relationships using a learning rule for hierarchical multi-label classification (HMC). Our detailed analysis demonstrates that this training strategy not only improves the model's confidence calibration but also retains additional information useful for downstream tasks such as fusion, prediction, and planning.

Hierarchical Insights: Exploiting Structural Similarities for Reliable 3D Semantic Segmentation

TL;DR

This work proposes a training strategy for a 3D LiDAR semantic segmentation model that learns structural relationships between classes through abstraction by implicitly modeling these relationships using a learning rule for hierarchical multi-label classification (HMC).

Abstract

Safety-critical applications such as autonomous driving require robust 3D environment perception algorithms capable of handling diverse and ambiguous surroundings. The predictive performance of classification models is heavily influenced by the dataset and the prior knowledge provided by the annotated labels. While labels guide the learning process, they often fail to capture the inherent relationships between classes that are naturally understood by humans. We propose a training strategy for a 3D LiDAR semantic segmentation model that learns structural relationships between classes through abstraction. This is achieved by implicitly modeling these relationships using a learning rule for hierarchical multi-label classification (HMC). Our detailed analysis demonstrates that this training strategy not only improves the model's confidence calibration but also retains additional information useful for downstream tasks such as fusion, prediction, and planning.
Paper Structure (13 sections, 5 equations, 4 figures, 5 tables)

This paper contains 13 sections, 5 equations, 4 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Literature
  3. Uncertainty Estimation in Classification Tasks
  4. Hierarchical Multi-Label Classification
  5. Methods
  6. Data and Label Hierarchy
  7. Hierarchical Multi-Label Classification
  8. Hierarchical Classification and Confidence Estimation
  9. Calibration Metrics for Semantic Segmentation
  10. Experimental Results
  11. Predictive Performance
  12. Calibration and Out of Distribution Robustness
  13. Conclusion

Figures (4)

  • Figure 1: The point cloud visualization of a nearby kid with a scooter (left: ground truth) from the SemanticKITTI Behley2019 dataset shows a stark difference between the models. A hierarchy-agnostic model (mid) misclassifies it as vegetation, whereas our hierarchy-aware model (right) correctly identifies it as dynamic.
  • Figure 2: Label hierarchy for the SemanticKITTI dataset: the original labels are leaf nodes, meta and binary classes are added accordingly. The colors denote the label colors as used in the dataset (except for the bicyclist class, whose color is adjusted for improved visibility).
  • Figure 3: Softmax probabilities $\sigma(\lambda)$ for classifications of classes motorcyclist (\ref{['subfig:probs_motorcyclist']}) and any (\ref{['subfig:probs_any']}). The colorscale is given below.
  • Figure 4: Qualitative samples from the SemanticKITTI Behley2019 dataset, showing the labels \ref{['subfig:label']} and the predictions and confidences of the vanilla Tang2020 (\ref{['subfig:flatpred']} & \ref{['subfig:flatconf']}), MCD Cortinhal2020 (\ref{['subfig:mcdpred']} & \ref{['subfig:mcdconf']}) and HMC model (\ref{['subfig:hmcpred']} & \ref{['subfig:hmcconf']}, $p_{\mathcal{H}}$ is used as confidence measure). The semantic class colors are depicted as in \ref{['fig:hierarchy']}, the confidence colorscale is given below.