TAMMs: Change Understanding and Forecasting in Satellite Image Time Series with Temporal-Aware Multimodal Models

TL;DR

SITS analysis traditionally treats temporal change understanding and future forecasting as separate tasks due to limited long-range temporal modeling. TAMMs unifies these tasks in a single MLLM-diffusion framework by introducing Temporal Adaptation Modules (TAM) to awaken temporal reasoning in frozen MLLMs and Semantic-Fused Control Injection (SFCI) to translate high-level change understanding into fine-grained generative control, accompanied by the Temporal Consistency Score (TCS) for evaluation. Empirical results show TAMMs achieves state-of-the-art performance on both temporal change description and temporally coherent future forecasting, with particularly strong gains in TCS, demonstrating improved alignment with historical dynamics. This work enables reasoning-driven, temporally coherent SITS forecasting and description, with practical implications for scalable remote sensing analysis and reproducible benchmarks.

Abstract

Temporal Change Description (TCD) and Future Satellite Image Forecasting (FSIF) are critical, yet historically disjointed tasks in Satellite Image Time Series (SITS) analysis. Both are fundamentally limited by the common challenge of modeling long-range temporal dynamics. To explore how to improve the performance of methods on both tasks simultaneously by enhancing long-range temporal understanding capabilities, we introduce **TAMMs**, the first unified framework designed to jointly perform TCD and FSIF within a single MLLM-diffusion architecture. TAMMs introduces two key innovations: Temporal Adaptation Modules (**TAM**) enhance frozen MLLM's ability to comprehend long-range dynamics, and Semantic-Fused Control Injection (**SFCI**) mechanism translates this change understanding into fine-grained generative control. This synergistic design makes the understanding from the TCD task to directly inform and improve the consistency of the FSIF task. Extensive experiments demonstrate TAMMs significantly outperforms state-of-the-art specialist baselines on both tasks. Our dataset can be found at https://huggingface.co/datasets/IceInPot/TAMMs .

Figures (10)

• Figure 1: Our TAMMs framework unifies temporal change understanding and future forecasting, using a temporal-aware MLLM to guide a diffusion model in a synergistic process.
• Figure 2: The TAMMs framework architecture. The temporal change understanding stage uses Temporal Adaptation Modules (TAM) to awaken the temporal reasoning of a frozen MLLM. The future forecasting stage then guides a frozen diffusion U-Net using our core Semantic-Fused Control Injection (SFCI) mechanism, which translates the MLLM's deep temporal understanding ($\mathbf{M}_t$) into multi-scale control signals. Critically, only the lightweight adapter components are trained.
• Figure 3: Temporal enhancement modules for the MLLM. PTE processes timestamps into learnable tokens to inject explicit temporal awareness into the visual feature stream. CTP (via prompting) provides high-level task guidance. Together, they enable the frozen MLLM to comprehend change narratives, which are then optimized via a composite loss.
• Figure 4: The Semantic-Fused Control Injection (SFCI) mechanism. SFCI guides the diffusion U-Net by dynamically fusing signals from two parallel streams: a Structural Path (green) that extracts low-level spatio-temporal dynamics, and a Semantic Path (gray) that translates the MLLM's high-level temporal understanding ($\mathbf{M}_t$). The resulting fused, multi-scale control signal is injected into the U-Net decoder to ensure temporally consistent forecasting.
• Figure 5: Qualitative comparison of temporal change description methods. Sequential images (a-c) are shown for each example, followed by outputs from baseline models and TAMMs. It can be seen that TAMMs has noticed many change details that other models have not paid attention to.