Spatio-Temporal driven Attention Graph Neural Network with Block Adjacency matrix (STAG-NN-BA) for Remote Land-use Change Detection

TL;DR

A hybrid attention method to learn the relative importance of irregular neighbors in remote sensing data is proposed and novel Spatio-Temporal driven Attention Graph Neural Network with Block Adjacency matrix (STAG-NN-BA) is proposed.

Abstract

Land-use monitoring is fundamental for spatial planning, particularly in view of compound impacts of growing global populations and climate change. Despite existing applications of deep learning in land use monitoring, standard convolutional kernels in deep neural networks limit the applications of these networks to the Euclidean domain only. Considering the geodesic nature of the measurement of the earth's surface, remote sensing is one such area that can benefit from non-Euclidean and spherical domains. For this purpose, we designed a novel Graph Neural Network architecture for spatial and spatio-temporal classification using satellite imagery to acquire insights into socio-economic indicators. We propose a hybrid attention method to learn the relative importance of irregular neighbors in remote sensing data. Instead of classifying each pixel, we propose a method based on Simple Linear Iterative Clustering (SLIC) image segmentation and Graph Attention Network. The superpixels obtained from SLIC become the nodes of our Graph Convolution Network (GCN). A region adjacency graph (RAG) is then constructed where each superpixel is connected to every other adjacent superpixel in the image, enabling information to propagate globally. Finally, we propose a Spatially driven Attention Graph Neural Network (SAG-NN) to classify each RAG. We also propose an extension to our SAG-NN for spatio-temporal data. Unlike regular grids of pixels in images, superpixels are irregular in nature and cannot be used to create spatio-temporal graphs. We introduce temporal bias by combining unconnected RAGs from each image into one supergraph. This is achieved by introducing block adjacency matrices resulting in novel Spatio-Temporal driven Attention Graph Neural Network with Block Adjacency matrix (STAG-NN-BA). SAG-NN and STAG-NN-BA outperform graph and non-graph baselines on Asia14 and C2D2 datasets efficiently.

Figures (9)

• Figure 1: Spatio-Temporal driven Attention Graph Neural Network with Block Adjacency matrix (STAG-NN-BA).
• Figure 2: Superpixel segmentation techniques on MNIST digit: 9.
• Figure 3: Region Adjacency Graphs (RAG) generation from SLIC, Quickshift and Felzenszwalb superpixels respectively.
• Figure 4: Superpixel segmentation techniques on image from Asia14 dataset. Felzenszwalbs's method and quickshift cannot segment perfectly built-up and barren land due to inherent complexities in satellite imagery. On the other hand, compact watershed poorly performed on grassy land. While SLIC works perfectly on satellite imagery.
• Figure 5: RAG generation from SLIC superpixels on image from Asia14 dataset (Satellite images courtesy Google Earth).