Explainable Earth Surface Forecasting under Extreme Events

TL;DR

A convolutional long short‐term memory‐based architecture is trained on the novel DeepExtremeCubes data set to showcase how the challenge of forecasting and understanding impacts on ecosystems can be met.

Abstract

With climate change-related extreme events on the rise, high dimensional Earth observation data presents a unique opportunity for forecasting and understanding impacts on ecosystems. This is, however, impeded by the complexity of processing, visualizing, modeling, and explaining this data. To showcase how this challenge can be met, here we train a convolutional long short-term memory-based architecture on the novel DeepExtremeCubes dataset. DeepExtremeCubes includes around 40,000 long-term Sentinel-2 minicubes (January 2016-October 2022) worldwide, along with labeled extreme events, meteorological data, vegetation land cover, and topography map, sampled from locations affected by extreme climate events and surrounding areas. When predicting future reflectances and vegetation impacts through kernel normalized difference vegetation index, the model achieved an R$^2$ score of 0.9055 in the test set. Explainable artificial intelligence was used to analyze the model's predictions during the October 2020 Central South America compound heatwave and drought event. We chose the same area exactly one year before the event as counterfactual, finding that the average temperature and surface pressure are generally the best predictors under normal conditions. In contrast, minimum anomalies of evaporation and surface latent heat flux take the lead during the event. A change of regime is also observed in the attributions before the event, which might help assess how long the event was brewing before happening. The code to replicate all experiments and figures in this paper is publicly available at https://github.com/DeepExtremes/txyXAI

Figures (7)

• Figure 1: A convolutional LSTM model was trained to forecast future Sentinel-2 reflectances and vegetation impacts given previous timesteps (augmented with ERA5 meteorology, elevation, and land cover). Explainable AI was then used to gather insights into the effects of extreme events on vegetation by comparing the model's attributions during event and comparable non-event situations. kNDVI: kernel normalized difference vegetation index.
• Figure 2: Overview of the spatio-temporal inputs (tensor $x_{st}$) for a single minicube sampled at location -99.47, 24.17 (China). Only a few timesteps are shown from 2021-08-19 to 2021-10-13.
• Figure 3: Map of the geographical distribution of the minicubes according to the subset to which they belong (red: train, green: validation, blue: test). Also, events lasting at least ten days have been marked on the map, along with their label ID, occurrence time, and duration.
• Figure 4: A Minicube at location 101.95°E, 46.97°N (Mongolia), from 2021-04-11 to 2021-06-15. Top three rows: ground truth RGB_next, B8A_next, and kNDVI_next. Bottom three rows: model predictions. Red outline: cloud_mask (labeled as "bad"). Minicube $L_1$: 0.0351, $L_2$: 0.0587, $R^2$: 0.9288, NNSE: 0.9335, bias: 0.0095
• Figure 5: Bar plot of the average attributions (over time, space, and minicubes) for top nine variables in tensor $x_{t}$ (top), $x_{s}$ (middle), and $x_{st}$ (bottom) of 24 minicubes affected by event 31833 (October 2020 central South America heatwave). Red bars represent the attribution for the model's outputs coinciding with the event, while the blue bars represent the attributions for the same period but one year before.