Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Feature-Augmented Deep Networks for Multiscale Building Segmentation in High-Resolution UAV and Satellite Imagery

Chintan B. Maniyar, Minakshi Kumar, Gengchen Mai

TL;DR

This paper addresses building footprint segmentation from high-resolution RGB imagery where spectral similarity and shadows degrade accuracy. It introduces a feature-augmented multiscale framework using guiding features derived from RGB channels (Sobel edges, VDVI, MBI, and PCA-derived PC1) and a dynamic Res-U-Net trained with a Combo-Loss, defined as $ComboLoss = BCE_{Loss} + \alpha \times Dice_{Loss}$ with $\alpha=1$, plus training policies including cyclical learning rates and SuperConvergence. On a held-out WorldView-3 image the method achieved $96.5\%$ accuracy, $F1$-score $=0.86$, and an IoU $=0.80$, outperforming RGB-based benchmarks. Key contributions include assembling a multiscale RGB UAV/satellite dataset (0.4–2.7 m), three band composites CB0/CB1/CB2, and demonstrating training-time reductions via CLR and SC.

Abstract

Accurate building segmentation from high-resolution RGB imagery remains challenging due to spectral similarity with non-building features, shadows, and irregular building geometries. In this study, we present a comprehensive deep learning framework for multiscale building segmentation using RGB aerial and satellite imagery with spatial resolutions ranging from 0.4m to 2.7m. We curate a diverse, multi-sensor dataset and introduce feature-augmented inputs by deriving secondary representations including Principal Component Analysis (PCA), Visible Difference Vegetation Index (VDVI), Morphological Building Index (MBI), and Sobel edge filters from RGB channels. These features guide a Res-U-Net architecture in learning complex spatial patterns more effectively. We also propose training policies incorporating layer freezing, cyclical learning rates, and SuperConvergence to reduce training time and resource usage. Evaluated on a held-out WorldView-3 image, our model achieves an overall accuracy of 96.5%, an F1-score of 0.86, and an Intersection over Union (IoU) of 0.80, outperforming existing RGB-based benchmarks. This study demonstrates the effectiveness of combining multi-resolution imagery, feature augmentation, and optimized training strategies for robust building segmentation in remote sensing applications.

Feature-Augmented Deep Networks for Multiscale Building Segmentation in High-Resolution UAV and Satellite Imagery

TL;DR

This paper addresses building footprint segmentation from high-resolution RGB imagery where spectral similarity and shadows degrade accuracy. It introduces a feature-augmented multiscale framework using guiding features derived from RGB channels (Sobel edges, VDVI, MBI, and PCA-derived PC1) and a dynamic Res-U-Net trained with a Combo-Loss, defined as with , plus training policies including cyclical learning rates and SuperConvergence. On a held-out WorldView-3 image the method achieved accuracy, -score , and an IoU , outperforming RGB-based benchmarks. Key contributions include assembling a multiscale RGB UAV/satellite dataset (0.4–2.7 m), three band composites CB0/CB1/CB2, and demonstrating training-time reductions via CLR and SC.

Abstract

Accurate building segmentation from high-resolution RGB imagery remains challenging due to spectral similarity with non-building features, shadows, and irregular building geometries. In this study, we present a comprehensive deep learning framework for multiscale building segmentation using RGB aerial and satellite imagery with spatial resolutions ranging from 0.4m to 2.7m. We curate a diverse, multi-sensor dataset and introduce feature-augmented inputs by deriving secondary representations including Principal Component Analysis (PCA), Visible Difference Vegetation Index (VDVI), Morphological Building Index (MBI), and Sobel edge filters from RGB channels. These features guide a Res-U-Net architecture in learning complex spatial patterns more effectively. We also propose training policies incorporating layer freezing, cyclical learning rates, and SuperConvergence to reduce training time and resource usage. Evaluated on a held-out WorldView-3 image, our model achieves an overall accuracy of 96.5%, an F1-score of 0.86, and an Intersection over Union (IoU) of 0.80, outperforming existing RGB-based benchmarks. This study demonstrates the effectiveness of combining multi-resolution imagery, feature augmentation, and optimized training strategies for robust building segmentation in remote sensing applications.
Paper Structure (24 sections, 18 equations, 11 figures, 7 tables)

This paper contains 24 sections, 18 equations, 11 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Non-learning based Building Segmentation Algorithms
  3. Spatial Information-based Building Segmentation
  4. Spectral Information-based Building Segmentation
  5. Knowledge-based Building Segmentation
  6. Learning-based Building Segmentation Algorithms
  7. Deep Learning Algorithms on Raw Spectral Bands
  8. Deep Learning Algorithms on Derived Feature Bands
  9. Objectives and Summary
  10. Materials and Methods
  11. Study Area
  12. Training Dataset Curation
  13. Generating Guiding Features
  14. Deep Neural Network - Architecture and Training
  15. Results
  16. ...and 9 more sections

Figures (11)

  • Figure 1: Inset map and snapshot highlighting Chandigarh City, India; the test site for this study.
  • Figure 2: Image and label pairs from each of the five datasets: (a) Inria Image Labelling Dataset, (b) Wuhan Housing Dataset, (c) Massachusetts Building Dataset, (d) CrowdAI Mapping Challenge Dataset, and (e) Open Cities AI Mapping Challenge Dataset
  • Figure 3: Methodology for extracting guiding features and generating different image composites for the dataset.
  • Figure 4: Three UAV images exhibiting the feature bands and Composite-0, Composite-1, and Composite-2 combinations. (a) Building class with road and open areas (b) Building class with shadow interference (c) Building class closely spaced and varying shapes
  • Figure 5: Three UAV images exhibiting the feature bands and Composite-0, Composite-1, and Composite-2 combinations. (a) Building class with road and open areas (b) Building class with shadow interference (c) Building class closely spaced and varying shapes
  • ...and 6 more figures