Monitoring of Urban Changes with multi-modal Sentinel 1 and 2 Data in Mariupol, Ukraine, in 2022/23

TL;DR

This study demonstrates that transfer learning of a multi-modal, time-series deep neural network (ERCNN-DRS) can enable urban-change monitoring in conflict zones using Sentinel-1/2 data when high-resolution ground truth is scarce. By transferring from an older VHR-ground-truth regime (2017–2020) to Mariupol (2017–2020 transfer period, monitored 2022–2023), and validating with Airbus Pléiades data, the authors show that the approach remains effective despite SAR data outages and limited data availability. The results indicate that optical observations drive detection performance more than SAR, with an ensemble of four transfer variants providing robust monitoring, and an ablation analysis confirming resilience to observation loss. The work provides a cost-efficient, high-temporal-resolution framework for monitoring urban changes in war and disaster zones where data are irregular or scarce.

Abstract

The ability to constantly monitor urban changes is of significant socio-economic interest, like detecting trends in urban expansion or tracking the vitality of urban areas. Especially in present conflict zones or disaster areas, such insights provide valuable information to keep track of the current situation. However, they are often subject to limited data availability in space and time. We built on our previous work, which used a transferred Deep Neural Network (DNN) operating on multi-modal Sentinel 1 and 2 data. In the current study, we have demonstrated and discussed its applicability in monitoring the present conflict zone of Mariupol, Ukraine, with high-temporal resolution Sentinel time series for the years 2022/23. A transfer to that conflict zone was challenging due to the limited availability of recent Very High Resolution (VHR) data. The current work had two objectives. First, transfer learning with older and publicly available VHR data was shown to be sufficient. That guaranteed the availability of more and less expensive data as time constraints were relaxed. Second, in an ablation study, we analyzed the effects of loss of observations to demonstrate the resiliency of our method. That was of particular interest due to the malfunctioning of Sentinel 1B shortly before the selected conflict. Our study demonstrated that urban change monitoring is possible for present conflict zones after transferring with older VHR data. It also indicated that, despite the multi-modal input, our method was more dependent on optical multispectral than Synthetic Aperture Radar (SAR) observations but resilient to loss of observations.

Figures (23)

• Figure 1: Flowchart of the transfer learning and monitoring process for the AoI of Mariupol. In blue are data and processing steps of our current work; gray denotes previous work.
• Figure 2: The two steps of generating the set of windows $\mathbb{W}_{i, j}$ for each tile with coordinates $i$ and $j$. The window predictions were used in combination with Maximum Pooling over Time to retrieve a combined prediction $\bm{y}^{max}_{i,j}$ during the transfer phase.
• Figure 3: Tiles for the AoI of Mariupol. The blue tiles covering 2017-2020 were used for training and validation, referred to as trainval dataset (164 in total). The 18 tiles in orange for 2022/23 were used for verification purposes, referred to as testing dataset. Geographic coordinates are in EPSG:4326 and tile coordinates are in $(y, x)$ dimensions. Background image © 2019/20 Google Earth, for reference only.
• Figure 4: Number of observations for each pixel within the AoI of Mariupol, separate for Sentinel 1 in ascending and descending orbit direction (left), and Sentinel 2 (right). Sentinel 1 has a range of [220, 546] and [324, 324] (no variation) for ascending and descending orbit directions, respectively. Sentinel 2 observations are within [166, 383]. Contours are shown for selected observation numbers.
• Figure 5: Windows and their observations. In our work, we combined Sentinel 1 (blue) and 2 (green) observations in a two-day ($\delta = 2 \textrm{ days}$) interval (grey). Highlighted in orange and discarded were windows with less than 35 ($\omega$) observations. A malfunction of Sentinel 1B occured on 23-12-2021 and the Russian invasion on 24-02-2022, as indicated on the timeline.