Classifying point clouds at the facade-level using geometric features and deep learning networks

TL;DR

This work tackles facade-level point-cloud classification to enable detailed 3D building models. It introduces an early-fusion approach that augments point coordinates with covariance-based geometric features, including $p$, $o$, $c$, $\lambda_i$, and the second eigenvector $\mathbf{e}_2$, and feeds them into PointNet and PointNet++ for improved segmentation. Random Forest-based feature importance guides the feature-selection process, and experiments on the TUM-FAÇADE dataset demonstrate that incorporating six features (XYZ+6F) yields substantial gains, with PointNet++ achieving up to $87.5\%$ overall accuracy on a test building. The results show that geometric features can compensate for local-structure capture gaps in deep nets, offering a practical route toward more accurate facade-level semantic segmentation, albeit with higher computational costs due to neighborhood searches.

Abstract

3D building models with facade details are playing an important role in many applications now. Classifying point clouds at facade-level is key to create such digital replicas of the real world. However, few studies have focused on such detailed classification with deep neural networks. We propose a method fusing geometric features with deep learning networks for point cloud classification at facade-level. Our experiments conclude that such early-fused features improve deep learning methods' performance. This method can be applied for compensating deep learning networks' ability in capturing local geometric information and promoting the advancement of semantic segmentation.

Figures (9)

• Figure 1: Point cloud classification with geometric features process for unseen datasets
• Figure 2: Workflow of geometric features extraction
• Figure 3: Proposed early-fusion in PointNet networks. Created based on Qi_2017_CVPR
• Figure 4: Experiments set-up
• Figure 5: Feature importance ranking in Random Forest