UB-FineNet: Urban Building Fine-grained Classification Network for Open-access Satellite Images

TL;DR

A new fine-grained classification network with Category Information Balancing Module (CIBM) and Contrastive Supervision (CS) technique is proposed to mitigate the problem of class imbalance and improve the classification robustness and accuracy.

Abstract

Fine classification of city-scale buildings from satellite remote sensing imagery is a crucial research area with significant implications for urban planning, infrastructure development, and population distribution analysis. However, the task faces big challenges due to low-resolution overhead images acquired from high altitude space-borne platforms and the long-tail sample distribution of fine-grained urban building categories, leading to severe class imbalance problem. To address these issues, we propose a deep network approach to fine-grained classification of urban buildings using open-access satellite images. A Denoising Diffusion Probabilistic Model (DDPM) based super-resolution method is first introduced to enhance the spatial resolution of satellite images, which benefits from domain-adaptive knowledge distillation. Then, a new fine-grained classification network with Category Information Balancing Module (CIBM) and Contrastive Supervision (CS) technique is proposed to mitigate the problem of class imbalance and improve the classification robustness and accuracy. Experiments on Hong Kong data set with 11 fine building types revealed promising classification results with a mean Top-1 accuracy of 60.45\%, which is on par with street-view image based approaches. Extensive ablation study shows that CIBM and CS improve Top-1 accuracy by 2.6\% and 3.5\% compared to the baseline method, respectively. And both modules can be easily inserted into other classification networks and similar enhancements have been achieved. Our research contributes to the field of urban analysis by providing a practical solution for fine classification of buildings in challenging mega city scenarios solely using open-access satellite images. The proposed method can serve as a valuable tool for urban planners, aiding in the understanding of economic, industrial, and population distribution.

Figures (15)

• Figure 1: Overview of the proposed building category classification network based on Google Earth satellite imagery.
• Figure 2: Schematic representation of the architecture of denoising diffusion probabilistic model (DDPM). (a) The diffusion process indicates the gradual process adding Gaussian noise to the target image ${\bm{y}}_{0}$ (from right to left), the reverse diffusion process depicts the gradual process of removing Gaussian noise from the source image ${\bm{y}}_{T}$(from right to left). (b) Reverse diffusion process from low-resolution image with trainable U-net based denoising network.
• Figure 3: Deviation correction module.
• Figure 4: Visualisation of features of different categories. (a) and (b) represent the intra-class distribution of spatial Euclidean distance of features produced by PCA before and after processing by the CIBM module, respectively, while (c) and (d) represent the intra-class distribution of spatial Euclidean distance of features produced by t-SNE before and after processing by the CIBM module, respectively.
• Figure 5: Comparison of proposed category information balanced module (CIBM) and commonly used methods. While the green part (c) is a common way to tackle the class imbalance problem in terms of the number of samples for each category, our proposed CIBM takes into account the cosine similarity between samples within each category by adding the blue area (a) and the orange area (b).