Towards Realistic Remote Sensing Dataset Distillation with Discriminative Prototype-guided Diffusion

TL;DR

This work tackles data-hungry remote sensing by distilling large RS datasets into a compact synthetic set while preserving discriminative power and privacy. It introduces discriminative prototype-guided diffusion (DPD), a latent-diffusion framework conditioned on CLIP text and guided by vision-language prototypes, optimized with a classification-consistency loss: $\\mathcal{L}_{total} =\\mathcal{L}_{D} + \\lambda\\mathcal{L}_{cls}$. Prototypes are extracted via vision-language clustering in the VAE latent space and enriched with aggregated captions from a large language model to form multimodal conditioning for diffusion. Experiments on UCM, AID, and NWPU-RESISC45 show that DPD consistently outperforms baselines across IPC settings and architectures, with notable gains at low data regimes, and demonstrate strong qualitative realism and diversity of distilled samples. This approach offers a practical, privacy-preserving route to scalable RS interpretation by enabling effective training on synthetic data while reducing storage and computation.

Abstract

Recent years have witnessed the remarkable success of deep learning in remote sensing image interpretation, driven by the availability of large-scale benchmark datasets. However, this reliance on massive training data also brings two major challenges: (1) high storage and computational costs, and (2) the risk of data leakage, especially when sensitive categories are involved. To address these challenges, this study introduces the concept of dataset distillation into the field of remote sensing image interpretation for the first time. Specifically, we train a text-to-image diffusion model to condense a large-scale remote sensing dataset into a compact and representative distilled dataset. To improve the discriminative quality of the synthesized samples, we propose a classifier-driven guidance by injecting a classification consistency loss from a pre-trained model into the diffusion training process. Besides, considering the rich semantic complexity of remote sensing imagery, we further perform latent space clustering on training samples to select representative and diverse prototypes as visual style guidance, while using a visual language model to provide aggregated text descriptions. Experiments on three high-resolution remote sensing scene classification benchmarks show that the proposed method can distill realistic and diverse samples for downstream model training. Code and pre-trained models are available online (https://github.com/YonghaoXu/DPD).

Figures (8)

• Figure 1: Overview of the motivation for this work. Through dataset distillation, large-scale remote sensing datasets can be condensed into compact synthetic datasets that enable models to achieve performance comparable to training on the original full dataset, while reducing storage, computation, and data privacy risks.
• Figure 2: Overview of the proposed classification-consistent diffusion training pipeline. Given a remote sensing image, a VAE encoder maps it into the latent space, where forward diffusion adds Gaussian noise. A U-Net then denoises the latent conditioned on CLIP text embeddings derived from the pseudo-caption, and the image is reconstructed by the VAE decoder. In addition to the diffusion loss, a pretrained classifier enforces semantic consistency through a classification loss.
• Figure 3: Illustration of the margin-based prototype selection, where the intra-cluster distance is minimized and the inter-cluster distance is maximized to obtain the prototype $z_{c,k}^*$.
• Figure 4: Qualitative comparison of randomly selected real images and distilled images generated by different dataset distillation methods on the UCM, AID, and NWPU datasets (IPC = 5). Note that all results are shown without manual "cherry picking".
• Figure 5: Visualization of distilled images across 21 classes on the UCM dataset (IPC = 5). Note that all results are shown without manual "cherry picking".