SENDAI: A Hierarchical Sparse-measurement, EfficieNt Data AssImilation Framework

TL;DR

SENDAI introduces a hierarchical sparse-measurement data assimilation framework that reconstructs full spatiotemporal fields from extremely sparse observations by combining simulation-derived priors with learned discrepancy corrections. A two-path scheme—a low-frequency pathway learned from simulation via Takens embedding and latent GAN alignment, plus a hierarchical high-frequency peeling path using coordinate-based implicit neural representations—enables accurate recovery of sharp boundaries and multi-scale dynamics under domain shifts. Across six globally distributed study sites and 64 sensors (about 1.5% coverage), SENDAI achieves substantial SSIM improvements over traditional baselines and recent neural methods, with Tarim Basin showing the largest gains due to strong topographic contrasts. The method is computationally lightweight, CPU-friendly, and operationally viable for real-time monitoring and resource-constrained deployments, with potential extensions to multivariate remote sensing tasks and broader domain adaptation challenges.

Abstract

Bridging the gap between data-rich training regimes and observation-sparse deployment conditions remains a central challenge in spatiotemporal field reconstruction, particularly when target domains exhibit distributional shifts, heterogeneous structure, and multi-scale dynamics absent from available training data. We present SENDAI, a hierarchical Sparse-measurement, EfficieNt Data AssImilation Framework that reconstructs full spatial states from hyper sparse sensor observations by combining simulation-derived priors with learned discrepancy corrections. We demonstrate the performance on satellite remote sensing, reconstructing MODIS (Moderate Resolution Imaging Spectroradiometer) derived vegetation index fields across six globally distributed sites. Using seasonal periods as a proxy for domain shift, the framework consistently outperforms established baselines that require substantially denser observations -- SENDAI achieves a maximum SSIM improvement of 185% over traditional baselines and a 36% improvement over recent high-frequency-based methods. These gains are particularly pronounced for landscapes with sharp boundaries and sub-seasonal dynamics; more importantly, the framework effectively preserves diagnostically relevant structures -- such as field topologies, land cover discontinuities, and spatial gradients. By yielding corrections that are more structurally and spectrally separable, the reconstructed fields are better suited for downstream inference of indirectly observed variables. The results therefore highlight a lightweight and operationally viable framework for sparse-measurement reconstruction that is applicable to physically grounded inference, resource-limited deployment, and real-time monitor and control.

Figures (21)

• Figure 1: Study sites for NDVI reconstruction experiments. (a) Global distribution of the six study sites, spanning: (b, f) Mediterranean, (c) continental, (d, e) arid, and (g) subtropical climates.
• Figure 2: SENDAI architecture. The LF pathway learns dominant dynamics from simulation and aligns to ground truth via a latent GAN. The HF pathway employs sequential peeling layers, each consisting of a sensor residual encoder and a coordinate-based INR decoder, to extract spectrally-distinct corrections.
• Figure 3: Hierarchical frequency peeling on a 1D NDVI slice from the Tarim Basin site. Top row presents single time-point reconstruction; bottom row presents spatiotemporal HF decomposition.
• Figure 4: Baseline reconstruction comparison for the Tarim Basin site. Red markers indicate sensor locations.
• Figure 5: (a) Sensor sensitivity results (from five independent runs) on the Tarim Basin site for the full SENDAI framework. (b) SENDAI Jr. reconstruction for the Central Valley site.