Assessing Building Heat Resilience Using UAV and Street-View Imagery with Coupled Global Context Vision Transformer

TL;DR

This paper tackles the challenge of scalable assessment of heat-relevant building attributes in dense urban areas of the Global South. It introduces a dual-modality framework that fuses UAV and street-view imagery through a coupled global context vision transformer (CGCViT) and links predicted building attributes to HotSat-1 $TIR$ measurements. Cross-view learning improves attribute classification over single-modality baselines, with vegetation, roofing materials (concrete/clay/wood), and higher roof brightness associated with lower $TIR$ values. Applied to Dar es Salaam, the approach reveals housing-material-related heat exposure disparities and provides a data-driven basis for targeted, equitable climate adaptation at the household level.

Abstract

Climate change is intensifying human heat exposure, particularly in densely built urban centers of the Global South. Low-cost construction materials and high thermal-mass surfaces further exacerbate this risk. Yet scalable methods for assessing such heat-relevant building attributes remain scarce. We propose a machine learning framework that fuses openly available unmanned aerial vehicle (UAV) and street-view (SV) imagery via a coupled global context vision transformer (CGCViT) to learn heat-relevant representations of urban structures. Thermal infrared (TIR) measurements from HotSat-1 are used to quantify the relationship between building attributes and heat-associated health risks. Our dual-modality cross-view learning approach outperforms the best single-modality models by up to $9.3\%$, demonstrating that UAV and SV imagery provide valuable complementary perspectives on urban structures. The presence of vegetation surrounding buildings (versus no vegetation), brighter roofing (versus darker roofing), and roofing made of concrete, clay, or wood (versus metal or tarpaulin) are all significantly associated with lower HotSat-1 TIR values. Deployed across the city of Dar es Salaam, Tanzania, the proposed framework illustrates how household-level inequalities in heat exposure - often linked to socio-economic disadvantage and reflected in building materials - can be identified and addressed using machine learning. Our results point to the critical role of localized, data-driven risk assessment in shaping climate adaptation strategies that deliver equitable outcomes.

Figures (8)

• Figure 1: Overview of the proposed CGCViT framework for extracting heat-relevant building attributes from paired UAV and SV imagery. Building footprints guide the extraction of cross-view image pairs, which are processed in parallel GCViT branches with global token integration. Predicted attributes include structural openness, number of floors, vegetation, wall/roofing material and brightness, and distance to surrounding buildings. These are statistically associated with HotSat-1 TIR values to inform heat mitigation strategies at building-level.
• Figure 2: Boxplots and scatter plots illustrate the relationships between HotSat-1 TIR values and building attributes identified in this study. They highlight a significant drop in mean HotsSat-1 TIR values for buildings surrounded by vegetation, as well as the important role of roofing materials and brightness.
• Figure 3: Study area—the Msimbazi River Delta in Dar es Salaam, Tanzania—covered by UAV imagery used throughout the analysis.
• Figure 4: Tricycle equipped with a GoPro Max used for collecting 360° SV imagery.
• Figure 5: Data acquisition process and spatial coverage of SV imagery. (a) Tricycle-based setup used for image capture. (b) Spatial distribution and visibility classification of buildings in the study area.