Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

ARCH2S: Dataset, Benchmark and Challenges for Learning Exterior Architectural Structures from Point Clouds

Ka Lung Cheung, Chi Chung Lee

TL;DR

The paper addresses the lack of fine-grained exterior 3D segmentation in outdoor point clouds and privacy-driven data gaps. It introduces ARCH2S, a semantically enriched, photorealistic 3D architectural models dataset with a semantic segmentation benchmark for exterior structures, drawing from real-world Hong Kong scenes and an open landscape. The dataset preparation includes model mining from the 3DBIT00 source, UV texture mapping, and sampling approximately 5M points per scene, with train/test splits and representative baselines (convolutional and transformer-based) evaluated using $OA$ and $mIoU$. The results highlight annotation inconsistencies, the superior performance of convolution-based methods over transformers on this task, and the challenges posed by imbalanced semantic distributions, underscoring the potential impact on BIM-enabled smart-city applications.

Abstract

Precise segmentation of architectural structures provides detailed information about various building components, enhancing our understanding and interaction with our built environment. Nevertheless, existing outdoor 3D point cloud datasets have limited and detailed annotations on architectural exteriors due to privacy concerns and the expensive costs of data acquisition and annotation. To overcome this shortfall, this paper introduces a semantically-enriched, photo-realistic 3D architectural models dataset and benchmark for semantic segmentation. It features 4 different building purposes of real-world buildings as well as an open architectural landscape in Hong Kong. Each point cloud is annotated into one of 14 semantic classes.

ARCH2S: Dataset, Benchmark and Challenges for Learning Exterior Architectural Structures from Point Clouds

TL;DR

The paper addresses the lack of fine-grained exterior 3D segmentation in outdoor point clouds and privacy-driven data gaps. It introduces ARCH2S, a semantically enriched, photorealistic 3D architectural models dataset with a semantic segmentation benchmark for exterior structures, drawing from real-world Hong Kong scenes and an open landscape. The dataset preparation includes model mining from the 3DBIT00 source, UV texture mapping, and sampling approximately 5M points per scene, with train/test splits and representative baselines (convolutional and transformer-based) evaluated using and . The results highlight annotation inconsistencies, the superior performance of convolution-based methods over transformers on this task, and the challenges posed by imbalanced semantic distributions, underscoring the potential impact on BIM-enabled smart-city applications.

Abstract

Precise segmentation of architectural structures provides detailed information about various building components, enhancing our understanding and interaction with our built environment. Nevertheless, existing outdoor 3D point cloud datasets have limited and detailed annotations on architectural exteriors due to privacy concerns and the expensive costs of data acquisition and annotation. To overcome this shortfall, this paper introduces a semantically-enriched, photo-realistic 3D architectural models dataset and benchmark for semantic segmentation. It features 4 different building purposes of real-world buildings as well as an open architectural landscape in Hong Kong. Each point cloud is annotated into one of 14 semantic classes.
Paper Structure (15 sections, 5 figures, 1 table)

This paper contains 15 sections, 5 figures, 1 table.

Table of Contents

  1. Introduction
  2. The ARCH2S Dataset
  3. Dataset Preparation
  4. Models Mining.
  5. Mesh Pre-processing.
  6. Point Sampling from Mesh.
  7. Benchmarks
  8. Train and Test Splits.
  9. Representative Baselines.
  10. Evaluation Metrics.
  11. Results and Discussion
  12. Challenges in Annotation.
  13. Learning Exterior Architectural Structures.
  14. Impact of Imbalanced Semantic Distribution.
  15. Conclusion

Figures (5)

  • Figure 1: Point cloud models from ARCH2S dataset. Different colors label different semantic classes.
  • Figure 2: Illustration of the stages of UV mapping with our mesh (Educational Facility). (a) The UV layout with 2D coordinates $(x, y)$, representing each point $p$ on the mesh's surface. (b) The 3D mesh in gray-scale, showing points $p$ in 3D space ($x, y, z)$ for texture projection. (c) The final textured mesh, with the UV map applied, displays detailed and colored surfaces.
  • Figure 3: Illustration of the random point sampling on our mesh. Plain mesh (top), wireframe display (middle), and corresponding sampled points (bottom).
  • Figure 4: Comparative analysis of raw point cloud data and semantic annotations in the Educational Facility, visualized on the annotation interface. The top row displays raw point cloud representations, while the bottom row shows the semantically annotated elements with various colors denoting different labeled classes. Red arrows indicate discrepancies where the semantic labels do not accurately match the actual built elements, highlighting areas of potential mislabeling.
  • Figure 5: The distribution of different semantic categories in ARCH2S dataset. Our dataset, consisting of 14 out of 22 semantic classes, covers a diverse range of common exterior architectural elements. Note that the "Garden" semantic label is reported only in the Commercial Centre point cloud. Moreover, point counts are on a logarithmic scale for the vertical axis.