TEA: Temporal Adaptive Satellite Image Semantic Segmentation

TL;DR

Crop parcel segmentation from Satellite Image Time Series often fails to generalize when temporal length varies. TEA introduces a temporal-adaptive framework that uses a global temporal teacher to guide a student processing adaptive-length inputs, aided by a temporal prototype alignment module and a data reconstruction task. A Length-Decayed IoU metric evaluates performance across sequence lengths. On PASTIS and Germany datasets, TEA achieves substantial gains over baselines across short to full sequences, confirming improved robustness and practical value for agricultural monitoring.

Abstract

Crop mapping based on satellite images time-series (SITS) holds substantial economic value in agricultural production settings, in which parcel segmentation is an essential step. Existing approaches have achieved notable advancements in SITS segmentation with predetermined sequence lengths. However, we found that these approaches overlooked the generalization capability of models across scenarios with varying temporal length, leading to markedly poor segmentation results in such cases. To address this issue, we propose TEA, a TEmporal Adaptive SITS semantic segmentation method to enhance the model's resilience under varying sequence lengths. We introduce a teacher model that encapsulates the global sequence knowledge to guide a student model with adaptive temporal input lengths. Specifically, teacher shapes the student's feature space via intermediate embedding, prototypes and soft label perspectives to realize knowledge transfer, while dynamically aggregating student model to mitigate knowledge forgetting. Finally, we introduce full-sequence reconstruction as an auxiliary task to further enhance the quality of representations across inputs of varying temporal lengths. Through extensive experiments, we demonstrate that our method brings remarkable improvements across inputs of different temporal lengths on common benchmarks. Our code will be publicly available.

Figures (8)

• Figure 1: Comparative crop mapping results on short sequences SITS, highlighting prior models in contrast to ours. L denotes the original sequence length. We evaluate segmentation performance on cropped sequences at length ratios of 20% and 60% as examples. The results expose shortcomings of prior methods, while showcasing our method’s superiority and robustness across varying temporal lengths.
• Figure 2: (a)The training pipeline of TEA. We adopt a teacher model which captures global temporal representations to supervise a temporal-adaptive student model. (b)Our Temporal Length Adaptive Network (TLAN) Insight. Model leverages the Temporal-Spatio Transformer backbone to process SITS inputs. To improve noise robustness and robustness across varying sequence lengths, we introduce an auxiliary task, data reconstruction, alongside a temporal prototype alignment mechanism. (c)Temporal prototype alignment module expand decision boundaries via clustering to enhance class discriminability for categories with weak temporal signatures.
• Figure 3: Visualization of segmentation results on PASTIS across different sequence lengths. Baseline denotes the TSViT model.
• Figure 4: Attention Weight Visualization of spatial. Both the horizontal and vertical axes represent the number of tokens. The color intensity represents the activation magnitude, with yellow at the top indicating the maximum value and blue at the bottom indicating the minimum value.
• Figure 5: Qualitative comparison between the baseline method and our TEA method on the PASTIS dataset. We mainly present the short-term time series testing results cropped at ratios of 20%, 40%, and 60% from the start of the sequence. The model input during testing is the same as during training, with a size of 24×24. We recombine the images of the same region into a 120×120 size image according to the cropping indices generated during data preprocessing.