Using street view imagery and deep generative modeling for estimating the health of urban forests

TL;DR

Urban forest health monitoring remains challenging at city scales due to labor intensity and limited sensing resources. The paper proposes a street-view–driven pipeline using a Pix2Pix conditional GAN to synthesize $NIR$ and thermal imagery from RGB inputs, augmented by embeddings for $I_{radiation}$, $S_{angle}$, and $I_{species}$ to estimate $NDVI$ and $CTD$ after canopy segmentation. Ground-truth validation is planned with onsite handheld multispectral and thermal sensors, enabling assessment of the surrogate imaging approach. If successful, the workflow could enable scalable, cost-effective urban forest monitoring using publicly available street-view data and standard inventories, with open access and documentation to support deployment.

Abstract

Healthy urban forests comprising of diverse trees and shrubs play a crucial role in mitigating climate change. They provide several key advantages such as providing shade for energy conservation, and intercepting rainfall to reduce flood runoff and soil erosion. Traditional approaches for monitoring the health of urban forests require instrumented inspection techniques, often involving a high amount of human labor and subjective evaluations. As a result, they are not scalable for cities which lack extensive resources. Recent approaches involving multi-spectral imaging data based on terrestrial sensing and satellites, are constrained respectively with challenges related to dedicated deployments and limited spatial resolutions. In this work, we propose an alternative approach for monitoring the urban forests using simplified inputs: street view imagery, tree inventory data and meteorological conditions. We propose to use image-to-image translation networks to estimate two urban forest health parameters, namely, NDVI and CTD. Finally, we aim to compare the generated results with ground truth data using an onsite campaign utilizing handheld multi-spectral and thermal imaging sensors. With the advent and expansion of street view imagery platforms such as Google Street View and Mapillary, this approach should enable effective management of urban forests for the authorities in cities at scale.

Figures (3)

• Figure 1: Current methods for monitoring health of urban forest and the scalability challenges associated with them due to spatial and temporal resolutions. Adapted from (akshitNatureSustain)
• Figure 2: Adding the meteorological variables as an embedding layer in Pix2Pix conditional GAN architecture to generate near-infrared from standard RGB images. The same methodology is used to generate thermal images.
• Figure 3: Using street view images from Google Street view to generate R,G, near infrared and thermal images, though previously trained models. The CTD and NDVI is calculated after segmentation of canopy as performed in (akshitIEEESensors