On the use of Graphs for Satellite Image Time Series

TL;DR

This paper surveys the use of graphs to model satellite image time series (SITS) by extending OBIA with spatio-temporal graphs. It presents a versatile pipeline for converting SITS into graphs, detailing object representation, spatial and spatio-temporal links, and fidelity/complexity trade-offs, then explores downstream analysis via expert insights, pattern mining, and graph-based learning (both unsupervised and supervised) as well as intrinsic regression tasks like imputation and forecasting. Through two case studies—land cover mapping and water resources forecasting—it demonstrates the benefits and limitations of graph-based approaches, including context-driven gains and computational bottlenecks dominated by graph construction. The discussion highlights interpretability, end-to-end graph learning, scalability, and extension to new modalities as key research directions with practical impact for Earth observation, while urging careful design choices and cross-disciplinary collaboration. Overall, the work shows that spatio-temporal graphs offer a flexible, interpretable intermediate representation that can preserve the structure of SITS and leverage GNNs for improved understanding and prediction of dynamic Earth processes.

Abstract

The Earth's surface is subject to complex and dynamic processes, ranging from large-scale phenomena such as tectonic plate movements to localized changes associated with ecosystems, agriculture, or human activity. Satellite images enable global monitoring of these processes with extensive spatial and temporal coverage, offering advantages over in-situ methods. In particular, resulting satellite image time series (SITS) datasets contain valuable information. To handle their large volume and complexity, some recent works focus on the use of graph-based techniques that abandon the regular Euclidean structure of satellite data to work at an object level. Besides, graphs enable modelling spatial and temporal interactions between identified objects, which are crucial for pattern detection, classification and regression tasks. This paper is an effort to examine the integration of graph-based methods in spatio-temporal remote-sensing analysis. In particular, it aims to present a versatile graph-based pipeline to tackle SITS analysis. It focuses on the construction of spatio-temporal graphs from SITS and their application to downstream tasks. The paper includes a comprehensive review and two case studies, which highlight the potential of graph-based approaches for land cover mapping and water resource forecasting. It also discusses numerous perspectives to resolve current limitations and encourage future developments.

Figures (17)

• Figure 1: Pipeline of a spatio-temporal graph used as an intermediate representation of satellite image time series (SITS) for different tasks. (a) SITS enable the study of phenomena occurring at the Earth's surface (\ref{['sec:SITSacqui']}). (b) The entities involved in the captured phenomena are identified and modeled in a spatio-temporal graph with their spatial and spatio-temporal relationships (\ref{['sec:graph_creation']}). The plot shows entities as black nodes, spatial edges as grey lines and spatio-temporal edges as blue dotted lines. This representation can then be exploited for the targeted application. (c) Graphs can serve as a visualization tool for experts (\ref{['sec:task_expert_analysis']}). The plot illustrates the normalized difference water index (NDWI) temporal profile of the entities, in the same way as in guttler2017, and suggests a soil covering event by water. (d) Graphs can help to detect predefined or frequent patterns (\ref{['sec:task_graph_pattern_mining']}). In the illustration, merger and continuation patterns, as defined in xu2021, are highlighted. (e) Graphs can be used for classification tasks in time and space (\ref{['sec:task_classification']}). (f) Graphs can efficiently handle the way entities dynamically interact with each other and thus provide anticipation of the future state of a component of the Earth system as part of a forecasting task (\ref{['sec:task_forecasting']}).
• Figure 2: Examples to illustrate graph terminology: (a) An undirected graph with node and edge attributes. The green shaded area, denoted by $\mathcal{N}(\mathrm{v}_5)$, depicts the neighborhood of node $\mathrm{v}_5$. (b) A multi-relational graph with weighted directed edges. The edge width represents the edge weight and orange and blue colored-edges represents two types of relationship.
• Figure 3: Various spatial mappings of a satellite image into objects, showed in random colors. (a-b) Original satellite image pixels plotted respectively along the spatial grid and in the CIELAB color space. (c-e) Spatial domain of objects defined respectively from a land register, a segmentation algorithm, and a spectral similarity criterion.
• Figure 4: Illustration of the endurantist and perdurantist paradigms for an object-based analysis of SITS. Acquisitions depict the evolution of Great Salt Lake water area; credit USGS. (a) According to endurantists, an object is wholly present at each moment of its life, with intrinsic properties as spectral signatures and spatial extent. The yellow outline shows the same object, but at different times and therefore with changing properties. (b) According to perdurantists, an object is a set of successive instantaneous temporal parts, occupying a spatio-temporal region. The yellow outline designates the whole spatio-temporal object where certain temporal parts are highlighted as slices of the object.
• Figure 5: Four ways to extract spatial object features with the region boundaries and satellite images as input. (a) Compute spectral, geometrical and textural features in a manually-defined process. (b) Extract deep features using a convolutional neural network on the padded or cropped region. (c) Apply graph neural network to the region's pixel graph. (d) Sample a fixed number of random pixels and compute statistical descriptors using a pixel-set encoder; adapted from garnot2020.