Unsupervised semantic segmentation of urban high-density multispectral point clouds

TL;DR

It was shown, that the GroupSP can semantically segment seven urban classes with oAcc of 95% and mIoU of 75% using only 0.004% of the available annotated points in the mapping assignment, and the multispectral information was examined; adding each new spectral channel improved the mIoU.

Abstract

The availability of highly accurate urban airborne laser scanning (ALS) data will increase rapidly in the future, especially as acquisition costs decrease, for example through the use of drones. Current challenges in data processing are related to the limited spectral information and low point density of most ALS datasets. Another challenge will be the growing need for annotated training data, frequently produced by manual processes, to enable semantic interpretation of point clouds. This study proposes to semantically segment new high-density (1200 points per square metre on average) multispectral ALS data with an unsupervised ground-aware deep clustering method GroupSP inspired by the unsupervised GrowSP algorithm. GroupSP divides the scene into superpoints as a preprocessing step. The neural network is trained iteratively by grouping the superpoints and using the grouping assignments as pseudo-labels. The predictions for the unseen data are given by over-segmenting the test set and mapping the predicted classes into ground truth classes manually or with automated majority voting. GroupSP obtained an overall accuracy (oAcc) of 97% and a mean intersection over union (mIoU) of 80%. When compared to other unsupervised semantic segmentation methods, GroupSP outperformed GrowSP and non-deep K-means. However, a supervised random forest classifier outperformed GroupSP. The labelling efforts in GroupSP can be minimal; it was shown, that the GroupSP can semantically segment seven urban classes (building, high vegetation, low vegetation, asphalt, rock, football field, and gravel) with oAcc of 95% and mIoU of 75% using only 0.004% of the available annotated points in the mapping assignment. Finally, the multispectral information was examined; adding each new spectral channel improved the mIoU. Additionally, echo deviation was valuable, especially when distinguishing ground-level classes.

Figures (11)

• Figure 1: FGI's in-house developed HeliALS-TW (a). The high-density multispectral ALS data was collected with three LiDAR scanners mounted on a helicopter (b).
• Figure 2: The high-density HeliALS data displays detailed geometric information, for example, tree branches (a) and balconies (b). The RGB colours of the figure are mapped to the reflectance values from scanners 1, 2, and 3, respectively.
• Figure 3: The scanned area is marked on the orthophoto and divided into unlabelled training and manually labelled test set primary tiles. Each tile covers a 200-metre by 200-metre area. Photo reference: "Orthophoto (c) Helsinki, Espoo, Vantaa, Kauniainen, Kirkkonummi, Kerava, Nurmijärvi, HSY, HSL and The Finnish Defence Forces 2023."
• Figure 4: Two examples of 200 m $\times$ 200 m primary tiles of urban multispectral point clouds created by combining three individual point clouds acquired with different wavelengths. Each point was assigned two additional reflectance values from the nearest neighbours acquired with the other two scanners. The RGB colours of the figure are mapped to the reflectance values from scanners 1, 2, and 3, respectively.
• Figure 5: Characteristic reflectance spectra for various materials and objects found in the Espoonlahti multispectral HeliALS dataset. The data has been obtained from the 25 boreal tree species spectral library hovi2017spectral and from the ASTER and the USGS spectral libraries baldridge2009asterkokaly2017usgs.