Spatiotemporal Satellite Image Downscaling with Transfer Encoders and Autoregressive Generative Models

TL;DR

Conventional downscaling of coarse satellite aerosol fields is challenged by nonlinear spatiotemporal structure and limited high-resolution data. The authors propose a two-stage transfer-learning framework combining a pretrained U-Net encoder with a diffusion-based downscaling model, validated by domain-similarity analysis between MERRA-2 and G5NR and a halo-Hann patch stitching scheme. The approach achieves high in-data accuracy (R^2 ≈0.65–0.94) and preserves spatial and temporal variability, with robust out-of-data performance and stable autoregressive reconstructions beyond the training window. This demonstrates a physically coherent, data-efficient pathway to generate long, high-resolution aerosol fields for environmental exposure assessment and monitoring.

Abstract

We present a transfer-learning generative downscaling framework to reconstruct fine resolution satellite images from coarse scale inputs. Our approach combines a lightweight U-Net transfer encoder with a diffusion-based generative model. The simpler U-Net is first pretrained on a long time series of coarse resolution data to learn spatiotemporal representations; its encoder is then frozen and transferred to a larger downscaling model as physically meaningful latent features. Our application uses NASA's MERRA-2 reanalysis as the low resolution source domain (50 km) and the GEOS-5 Nature Run (G5NR) as the high resolution target (7 km). Our study area included a large area in Asia, which was made computationally tractable by splitting into two subregions and four seasons. We conducted domain similarity analysis using Wasserstein distances confirmed minimal distributional shift between MERRA-2 and G5NR, validating the safety of parameter frozen transfer. Across seasonal regional splits, our model achieved excellent performance (R2 = 0.65 to 0.94), outperforming comparison models including deterministic U-Nets, variational autoencoders, and prior transfer learning baselines. Out of data evaluations using semivariograms, ACF/PACF, and lag-based RMSE/R2 demonstrated that the predicted downscaled images preserved physically consistent spatial variability and temporal autocorrelation, enabling stable autoregressive reconstruction beyond the G5NR record. These results show that transfer enhanced diffusion models provide a robust and physically coherent solution for downscaling a long time series of coarse resolution images with limited training periods. This advancement has significant implications for improving environmental exposure assessment and long term environmental monitoring.

Figures (11)

• Figure 1: Overview of the downscaling framework: (A) A small U-Net model $f^{\mathrm{UNet}}_{\Omega}$ is pretrained on MERRA-2 to learn long-range spatiotemporal structure; (B) The encoder $\phi_{\psi}$ is frozen from model (A), extracted, and transferred to generate inputs for G5NR sequences; (C) The large downscaling denoising diffusion probabilistic model $f^{\mathrm{DDPM}}_{\Theta}$ predicts high-resolution G5NR dust-extinction image $\hat{y}_{i,j,t+1}$ for day $t{+}1$.
• Figure 2: Top: image of true G5NR (left) and MERRA-2 (right) chosen from a single day in Season 3 and Area 0 (Afghanistan and Kyrgyzstan) with their corresponding latitude and longitude in y and x axes. Bottom: image of single-day true G5NR (left) and MERRA-2 (right) chosen from season 3 and Area 1 (Gulf countries and Djibouti)
• Figure 3: Pixel-wise temporal-lag diagnostics for Season 3 in both regions. Each panel shows within-dataset temporal persistence of true G5NR and MERRA-2 dust-extinction fields. Top row: pixel-wise $R^2$ between daily fields separated by 1–10 day lags; bottom row: corresponding RMSE values. Higher $R^2$ indicates stronger temporal persistence, while increasing RMSE reflects faster temporal variability among images across time lags.
• Figure 4: Native-resolution spatial structure during the overlap period of True G5NR and True MERRA-2. Semivariograms are computed per day and averaged within the Season$\times$Area split; spherical fits summarize the averaged curves.
• Figure 5: Native-resolution temporal structure during the overlap period for Season 3, A0. ACF and PACF of the area-mean AOD for lags 1–16 days, computed within the Season$\times$Area split. Faster ACF decay for G5NR indicates stronger short-lag variability; PACF mass concentrates at the first few lags for both datasets.