ZAHA: Introducing the Level of Facade Generalization and the Large-Scale Point Cloud Facade Semantic Segmentation Benchmark Dataset

TL;DR

ZAHA addresses the fragmentation of 3D facade semantics by introducing Level of Facade Generalization (LoFG), a hierarchical, standards-aligned taxonomy that unifies facade element labels across CityGML, IFC, and AAT frameworks. It presents ZAHA, the largest real-world 3D facade benchmark with 601 million annotated points from 66 facades, annotated at LoFG3 (15 classes) and LoFG2 (5 classes), enabling rigorous cross-method evaluation. Through experiments with PointNet, PointNet++, Point Transformer, and DGCNN, the study shows that planar, well-represented classes segment well, while complex, underrepresented elements (e.g., deco) remain challenging, and that performance strongly depends on the chosen LoFG level. The work concludes with implications for 3D facade reconstruction and urban digital twins, and plans to host a 3D facade segmentation challenge to foster further progress and standardized benchmarking.

Abstract

Facade semantic segmentation is a long-standing challenge in photogrammetry and computer vision. Although the last decades have witnessed the influx of facade segmentation methods, there is a lack of comprehensive facade classes and data covering the architectural variability. In ZAHA, we introduce Level of Facade Generalization (LoFG), novel hierarchical facade classes designed based on international urban modeling standards, ensuring compatibility with real-world challenging classes and uniform methods' comparison. Realizing the LoFG, we present to date the largest semantic 3D facade segmentation dataset, providing 601 million annotated points at five and 15 classes of LoFG2 and LoFG3, respectively. Moreover, we analyze the performance of baseline semantic segmentation methods on our introduced LoFG classes and data, complementing it with a discussion on the unresolved challenges for facade segmentation. We firmly believe that ZAHA shall facilitate further development of 3D facade semantic segmentation methods, enabling robust segmentation indispensable in creating urban digital twins.

Figures (5)

• Figure 1: ZAHA: The dataset comprising 66 facades of various architectural style yielding over 601 million facade-level annotated terrestrial point clouds with distinct 15 facade-relevant classes.
• Figure 2: Selected facades of ZAHA. The dataset comprises 15 classes distributed over a diverse facade styles and functions, presenting a versatile training and testing segmentation scenario.
• Figure 3: Level of Facade Generalization (LoFG): Primary 15 classes (blue) representing LoFG 3 at the finest level of generalization. These aggregate into coarser LoFG 2 (purple) based on their geometrical and semantical representation; The LoFG 1 (green) is designed as an abstract class allowing to group elements describing facade. Such hierarchical and harmonized representation offers adaptability to the downstream tasks of the built environment, addresses imbalanced facade datasets (long-tail), and allows for various but unified methods' testing.
• Figure 5: LoFG3 inference on the test set comprising residential (top), underpass and cultural heritage (middle), and university buildings (bottom).
• Figure 6: LoFG2 inference on the test set comprising residential (top), underpass and cultural heritage (middle), and university buildings (bottom); color-coding according to the most prominent merged sub-class.