Texture2LoD3: Enabling LoD3 Building Reconstruction With Panoramic Images

TL;DR

Texture2LoD3 addresses the challenge of LoD3 building reconstruction by leveraging panoramic street-view images and widely available low-detail semantic CityGML-like models as priors. The method couples panorama-to-model global matching, panorama auto-rectification, facade segmentation, quadrilateral fitting, and ray-casting texturing to produce georeferenced, watertight LoD3 facades from unrectified imagery. A new ReLoD3 benchmark provides synchronized 3D models, panoramas, and ground-truth textures to quantify improvements, including about an 11% uplift in facade segmentation accuracy. The approach enables scalable LoD3 reconstruction with practical impact for applications such as solar potential estimation and autonomous driving simulations.

Abstract

Despite recent advancements in surface reconstruction, Level of Detail (LoD) 3 building reconstruction remains an unresolved challenge. The main issue pertains to the object-oriented modelling paradigm, which requires georeferencing, watertight geometry, facade semantics, and low-poly representation -- Contrasting unstructured mesh-oriented models. In Texture2LoD3, we introduce a novel method leveraging the ubiquity of 3D building model priors and panoramic street-level images, enabling the reconstruction of LoD3 building models. We observe that prior low-detail building models can serve as valid planar targets for ortho-rectifying street-level panoramic images. Moreover, deploying segmentation on accurately textured low-level building surfaces supports maintaining essential georeferencing, watertight geometry, and low-poly representation for LoD3 reconstruction. In the absence of LoD3 validation data, we additionally introduce the ReLoD3 dataset, on which we experimentally demonstrate that our method leads to improved facade segmentation accuracy by 11% and can replace costly manual projections. We believe that Texture2LoD3 can scale the adoption of LoD3 models, opening applications in estimating building solar potential or enhancing autonomous driving simulations. The project website, code, and data are available here: https://wenzhaotang.github.io/Texture2LoD3/.

Figures (12)

• Figure 1: Texture2LoD3 proposes leveraging ubiquitous street-level images and low-level building models for accurate ortho-texturing (left): Enabling accurate semantic segmentation (center) and facade-rich 3 reconstruction (right).
• Figure 2: Overview of the proposed Texture2LoD3 method: The method commences with global matching of georeferenced panorama images and low-level 3D models. In the top branch, 3D target facade surfaces are simplified, while in the bottom branch panoramic images are rectified and building facade instances are extracted. Subsequently, fine object-to-object matching and projection is performed to the simplified 3D model surface. Quadrilateral fitting and image-to-plane ray casting ensure accurate ortho-rectified 3D texture, enabling accurate facade elements segmentation and 3 reconstruction.
• Figure 3: (Left) Original surface with multiple triangular faces. (Right) Fitted quadrilateral representation with re-triangulation along the diagonal (dashed purple), preserving facade shape.
• Figure 4: Semantic-SAM generates unlabeled instance masks, which are then passed to a CLIP encoder for semantic filtering. We retain masks classified as building facade.
• Figure 5: Building facade after filtering out extraneous parts, e.g., eave masks, and a schematic view quadrilateral fitting extracting the four corner points based on the refined mask.