SatelliteCalculator: A Multi-Task Vision Foundation Model for Quantitative Remote Sensing Inversion

TL;DR

This work tackles the lack of foundation models for quantitative remote sensing inversion by introducing SatelliteCalculator, a prompt-guided, multi-task vision foundation model. It combines a frozen Swin Transformer backbone with per-task cross-attentive adapters and lightweight MLP decoders to jointly estimate eight ecological indicators from Sentinel-2 data, trained on a large-scale, physically defined Open-Canopy-derived dataset. The approach achieves competitive accuracy across tasks while reducing inference cost, and it demonstrates scalable adaptability to new inversion targets via trainable task prompts. The study provides a scalable framework for physically interpretable multi-task regression in remote sensing, with strong implications for environmental monitoring and policy-relevant ecological estimation.

Abstract

Quantitative remote sensing inversion plays a critical role in environmental monitoring, enabling the estimation of key ecological variables such as vegetation indices, canopy structure, and carbon stock. Although vision foundation models have achieved remarkable progress in classification and segmentation tasks, their application to physically interpretable regression remains largely unexplored. Furthermore, the multi-spectral nature and geospatial heterogeneity of remote sensing data pose significant challenges for generalization and transferability. To address these issues, we introduce SatelliteCalculator, the first vision foundation model tailored for quantitative remote sensing inversion. By leveraging physically defined index formulas, we automatically construct a large-scale dataset of over one million paired samples across eight core ecological indicators. The model integrates a frozen Swin Transformer backbone with a prompt-guided architecture, featuring cross-attentive adapters and lightweight task-specific MLP decoders. Experiments on the Open-Canopy benchmark demonstrate that SatelliteCalculator achieves competitive accuracy across all tasks while significantly reducing inference cost. Our results validate the feasibility of applying foundation models to quantitative inversion, and provide a scalable framework for task-adaptive remote sensing estimation.

Figures (5)

• Figure 1: Overview of the SatelliteCalculator framework. The model takes a multi-band Sentinel-2 VHR image and a task prompt as input, and performs quantitative inversion through four components: (1) prompt embedding module, (2) Swin Transformer feature extractor, (3) Cross-attentive adapter, and (4) Task-specific decoder.
• Figure 2: Comparison of canopy height inversion results across different models. From left to right: Sentinel-2 VHR input, groundtruth, estimations from SatelliteCalculator, SWIN, PVTv2, PCPVT, UNet, HVIT, DeepLabv3, and ViT-B.
• Figure 3: Multi-task inversion results on four Sentinel-2 scenes. We show groundtruth (top) and SatelliteCalculator estimations (bottom) for five spectral indices and three structural variables.
• Figure 4: Efficiency comparison across four decoder architectures (MLP, UNet, ResNet, and Transformer). Each axis represents one of four evaluation metrics where outer regions indicate better performance. The chart highlights trade-offs between model complexity and efficiency, showing that MLP achieves optimal inference and memory usage.
• Figure 5: Effect of MLP layer depth on decoder performance. We report the variation of MAE, RMSE, $R^2$, and PSNR as the MLP depth increases from 1 to 10. The results show that using 4 layers achieves the best overall performance, while deeper architectures tend to overfit and degrade generalization.