Towards Explainable LiDAR Point Cloud Semantic Segmentation via Gradient Based Target Localization

TL;DR

pGS-CAM addresses the interpretability gap in LiDAR point cloud semantic segmentation by extending Grad-CAM to generate per-point saliency maps across intermediate activations. It aggregates gradients in a subset of points to produce layerwise explanations and uses KDTree upsampling to map heatmaps back to the full point cloud, enabling insightful visualizations across KPConv and RandLA-Net on datasets such as SemanticKITTI, Paris-Lille3D, and DALES. The work includes qualitative and quantitative analyses, dimensionality-reduction comparisons, counterfactual explanations, and point-drop robustness, with code released for reproducibility. Overall, pGS-CAM provides a granular, gradient-driven lens into how 3D segmentation models allocate attention to points, aiding debugging, model comparison, and reliability in real-world deployments.

Abstract

Semantic Segmentation (SS) of LiDAR point clouds is essential for many applications, such as urban planning and autonomous driving. While much progress has been made in interpreting SS predictions for images, interpreting point cloud SS predictions remains a challenge. This paper introduces pGS-CAM, a novel gradient-based method for generating saliency maps in neural network activation layers. Inspired by Grad-CAM, which uses gradients to highlight local importance, pGS-CAM is robust and effective on a variety of datasets (SemanticKITTI, Paris-Lille3D, DALES) and 3D deep learning architectures (KPConv, RandLANet). Our experiments show that pGS-CAM effectively accentuates the feature learning in intermediate activations of SS architectures by highlighting the contribution of each point. This allows us to better understand how SS models make their predictions and identify potential areas for improvement. Relevant codes are available at https://github.com/geoai4cities/pGS-CAM.

Figures (8)

• Figure 1: pGS-CAM for single point (white dot) and class Road, points corresponding to subset of class Person, points corresponding to all points of class Car. All heatmaps are obtained with respect to the final activation layers.
• Figure 2: pGS-CAM heatmaps for SemanticKITTI car class and RandLA-Net architecture.
• Figure 3: pGS-CAM heatmaps for Paris-Lille3D car class and KPConv architecture.
• Figure 4: pGS-CAM heatmaps for DALES building class and KPConv architecture.
• Figure 5: pGS-CAM heatmaps comparison with PCA and TSNE plots for initial activation layer and target class $car$. Notice the highlights of vegetation points in heatmap which is reflected by the cluttering of car and vegetation in PCA and TSNE plots.