Spatio-Temporal Transformers for Long-Term NDVI Forecasting

TL;DR

The paper tackles long-term NDVI forecasting from heterogeneous satellite time series, focusing on the Mediterranean where multi-scale spatial and multi-year temporal dynamics interact. It proposes STT-LTF, a self-supervised spatio-temporal transformer that ingests multi-scale spatial patches, cyclical temporal encodings, and geographic context to predict arbitrary future NDVI values in a single forward pass. Key contributions include a comprehensive spatio-temporal embedding framework, three masking strategies for robust SSL training on unlabeled Landsat data, and state-of-the-art next-year NDVI predictions (MAE 0.0328, R^2 0.8412) outperforming traditional statistical, CNN-based, LSTM, and standard Transformer baselines. The approach enables direct horizon forecasts with irregular sampling and variable horizons, supporting climate impact assessment and ecosystem monitoring in complex landscapes, with potential extensions to multi-modal data and geographic transferability.

Abstract

Long-term satellite image time series (SITS) analysis in heterogeneous landscapes faces significant challenges, particularly in Mediterranean regions where complex spatial patterns, seasonal variations, and multi-decade environmental changes interact across different scales. This paper presents the Spatio-Temporal Transformer for Long Term Forecasting (STT-LTF ), an extended framework that advances beyond purely temporal analysis to integrate spatial context modeling with temporal sequence prediction. STT-LTF processes multi-scale spatial patches alongside temporal sequences (up to 20 years) through a unified transformer architecture, capturing both local neighborhood relationships and regional climate influences. The framework employs comprehensive self-supervised learning with spatial masking, temporal masking, and horizon sampling strategies, enabling robust model training from 40 years of unlabeled Landsat imagery. Unlike autoregressive approaches, STT-LTF directly predicts arbitrary future time points without error accumulation, incorporating spatial patch embeddings, cyclical temporal encoding, and geographic coordinates to learn complex dependencies across heterogeneous Mediterranean ecosystems. Experimental evaluation on Landsat data (1984-2024) demonstrates that STT-LTF achieves a Mean Absolute Error (MAE) of 0.0328 and R^2 of 0.8412 for next-year predictions, outperforming traditional statistical methods, CNN-based approaches, LSTM networks, and standard transformers. The framework's ability to handle irregular temporal sampling and variable prediction horizons makes it particularly suitable for analysis of heterogeneous landscapes experiencing rapid ecological transitions.

Figures (6)

• Figure 1: Study area characteristics in the southeastern Mediterranean basin.
• Figure 2: Time series of NDVI values (1985–2024) for three samples, illustrating seasonal variability and long-term trends. Source: faran2025sstltp
• Figure 3: Self-supervised learning framework for NDVI time-series prediction. The model processes $T$ historical satellite observations with spatial patches, each containing NDVI, month, and year and geographic context to predict future NDVI values at target time $T+\Delta$ using MAE loss against ground truth NDVI from actual satellite observations.
• Figure 4: Architecture of the proposed STT-LTF model, combining spatial, temporal, and geographic features. Input spatio-temporal patches undergo masking strategies before multi-channel embedding (spatial, temporal, coordinates, positional), transformer processing, and regression to predict target NDVI values through self-supervised learning.
• Figure 5: MAE loss against past sequence length for different future prediction horizons ($\Delta t$) with spatial patch size 1. Results show that MAE loss decreases with longer historical sequences and shorter prediction horizons. Adopted from faran2025sstltp.