Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Exploiting Semantic Scene Reconstruction for Estimating Building Envelope Characteristics

Chenghao Xu, Malcolm Mielle, Antoine Laborde, Ali Waseem, Florent Forest, Olga Fink

TL;DR

BuildNet3D, a novel framework to estimate geometric building characteristics from 2D image inputs by integrating SDF-based representation with semantic modality, recovers fine-grained 3D geometry and semantics of building envelopes, which are then used to automatically extract building characteristics.

Abstract

Achieving the EU's climate neutrality goal requires retrofitting existing buildings to reduce energy use and emissions. A critical step in this process is the precise assessment of geometric building envelope characteristics to inform retrofitting decisions. Previous methods for estimating building characteristics, such as window-to-wall ratio, building footprint area, and the location of architectural elements, have primarily relied on applying deep-learning-based detection or segmentation techniques on 2D images. However, these approaches tend to focus on planar facade properties, limiting their accuracy and comprehensiveness when analyzing complete building envelopes in 3D. While neural scene representations have shown exceptional performance in indoor scene reconstruction, they remain under-explored for external building envelope analysis. This work addresses this gap by leveraging cutting-edge neural surface reconstruction techniques based on signed distance function (SDF) representations for 3D building analysis. We propose BuildNet3D, a novel framework to estimate geometric building characteristics from 2D image inputs. By integrating SDF-based representation with semantic modality, BuildNet3D recovers fine-grained 3D geometry and semantics of building envelopes, which are then used to automatically extract building characteristics. Our framework is evaluated on a range of complex building structures, demonstrating high accuracy and generalizability in estimating window-to-wall ratio and building footprint. The results underscore the effectiveness of BuildNet3D for practical applications in building analysis and retrofitting.

Exploiting Semantic Scene Reconstruction for Estimating Building Envelope Characteristics

TL;DR

BuildNet3D, a novel framework to estimate geometric building characteristics from 2D image inputs by integrating SDF-based representation with semantic modality, recovers fine-grained 3D geometry and semantics of building envelopes, which are then used to automatically extract building characteristics.

Abstract

Achieving the EU's climate neutrality goal requires retrofitting existing buildings to reduce energy use and emissions. A critical step in this process is the precise assessment of geometric building envelope characteristics to inform retrofitting decisions. Previous methods for estimating building characteristics, such as window-to-wall ratio, building footprint area, and the location of architectural elements, have primarily relied on applying deep-learning-based detection or segmentation techniques on 2D images. However, these approaches tend to focus on planar facade properties, limiting their accuracy and comprehensiveness when analyzing complete building envelopes in 3D. While neural scene representations have shown exceptional performance in indoor scene reconstruction, they remain under-explored for external building envelope analysis. This work addresses this gap by leveraging cutting-edge neural surface reconstruction techniques based on signed distance function (SDF) representations for 3D building analysis. We propose BuildNet3D, a novel framework to estimate geometric building characteristics from 2D image inputs. By integrating SDF-based representation with semantic modality, BuildNet3D recovers fine-grained 3D geometry and semantics of building envelopes, which are then used to automatically extract building characteristics. Our framework is evaluated on a range of complex building structures, demonstrating high accuracy and generalizability in estimating window-to-wall ratio and building footprint. The results underscore the effectiveness of BuildNet3D for practical applications in building analysis and retrofitting.

Paper Structure

This paper contains 18 sections, 15 equations, 4 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Building Characteristics Estimation
  4. 3D Object Reconstruction
  5. Preliminary
  6. SDF-Based Volume Rendering
  7. Neural Surface Reconstruction Loss
  8. Methodology
  9. Semantic Building Reconstruction
  10. Building Characteristics Estimation
  11. Window-to-Wall Ratio
  12. Building Footprint
  13. BuildNet3D Dataset
  14. Experiments
  15. Implementation Details
  16. ...and 3 more sections

Figures (4)

  • Figure 1: Architecture of the BuildNet3D framework for estimating building characteristics from 2D image inputs. The 2D color images with corresponding semantic images are utilized to reconstruct a semantic building in 3D.
  • Figure 2: Architecture of reconstruction module for 3D semantic building envelope using SDF-based representation.
  • Figure 3: The pipeline of our improved 2D-semantics-based baseline method for estimating WWR of entire building envelopes with consistent scaling factors across all facades.
  • Figure 4: Rendered examples showcasing color, semantic labels, surface normals, and depth images for various structures, including a synthetic skyscraper, a synthetic L-shaped building, and realistic geometrically complex structures, arranged in order of increasing geometric complexity. BuildNet3D utilizes only the color and semantic images.