ChangeDiff: A Multi-Temporal Change Detection Data Generator with Flexible Text Prompts via Diffusion Model

TL;DR

ChangeDiff addresses data scarcity in semantic change detection by introducing a two-stage diffusion-based data generator that first creates temporally continuous layouts from text prompts and then renders images conditioned on those layouts. It introduces multi-class distribution-guided text prompts (MCDG-TP) and a class distribution refinement loss to align text cues with model attention, followed by a layout-to-image diffusion step to produce realistic multi-temporal pairs without paired data. The approach supports three time-varying modes to simulate progressive changes and demonstrates significant improvements in CD performance and transferability across multiple datasets, with qualitative evidence of higher realism and fewer artifacts. This work reduces annotation burden and enhances cross-domain CD applications, providing code for reproducibility and broader adoption.

Abstract

Data-driven deep learning models have enabled tremendous progress in change detection (CD) with the support of pixel-level annotations. However, collecting diverse data and manually annotating them is costly, laborious, and knowledge-intensive. Existing generative methods for CD data synthesis show competitive potential in addressing this issue but still face the following limitations: 1) difficulty in flexibly controlling change events, 2) dependence on additional data to train the data generators, 3) focus on specific change detection tasks. To this end, this paper focuses on the semantic CD (SCD) task and develops a multi-temporal SCD data generator ChangeDiff by exploring powerful diffusion models. ChangeDiff innovatively generates change data in two steps: first, it uses text prompts and a text-to-layout (T2L) model to create continuous layouts, and then it employs layout-to-image (L2I) to convert these layouts into images. Specifically, we propose multi-class distribution-guided text prompts (MCDG-TP), allowing for layouts to be generated flexibly through controllable classes and their corresponding ratios. Subsequently, to generalize the T2L model to the proposed MCDG-TP, a class distribution refinement loss is further designed as training supervision. %For the former, a multi-classdistribution-guided text prompt (MCDG-TP) is proposed to complement via controllable classes and ratios. To generalize the text-to-image diffusion model to the proposed MCDG-TP, a class distribution refinement loss is designed as training supervision. For the latter, MCDG-TP in three modes is proposed to synthesize new layout masks from various texts. Our generated data shows significant progress in temporal continuity, spatial diversity, and quality realism, empowering change detectors with accuracy and transferability. The code is available at https://github.com/DZhaoXd/ChangeDiff

Figures (5)

• Figure 1: Multi-temporal semantic change synthetic data synthesized by our ChangeDiff, which is trained on the sparsely labeled semantic change detection SECONDyang2021asymmetric dataset. It describes an area under construction, where man-made facilities are gradually being completed. ChangeDiff takes text prompts as input, generates semantic events, and specifies changes in a controllable manner by modifying the text prompts.
• Figure 2: An overview of the proposed ChangeDiff, which consists of two components, i.e., changing layout synthesis (CLS) and changing image synthesis (CIS). In the CLS: I. The text-to-layout (T2L) model is fine-tuned on the target data via MCDG-TP; II. Text with amplified class ratios is fed into the fine-tuned T2L model to generate the completed layouts; III. Taking the completed layout as a reference, texts with different class compositions are fed into the fine-tuned T2L model to synthesize temporally changed layouts. In the CIS: New images aligned with the changed layouts are synthesized via a layout-to-image model fine-tuned to the target data.
• Figure 3: Comparison of data augmentation methods on two SCD datasets against competing approaches. We reproduce the results of Changen on the two SCD datasets, not using other segmentation datasets.
• Figure 4: Visual comparison of different augmentation methods for semantic change detection. a) Training data in the SECOND dataset. The white areas in GT are not labeled. b) Self-Pair seo2023self: Pasting objects from other patches onto the target image to create changes. c) Changen zheng2023scalable: Synthesizing images from a given sparse-labeled layout, and d) synthesizing images from a randomly filled layout. e) Ours: Synthesizing images from our synthetic layout. The red boxes in b), c), and d) show the low image quality and unknown semantics due to the lack of reasonable layout.
• Figure 5: Impact of different pre-train on model performance in the SECOND dataset. We compare: 1) pre-training with ImageNet classification data, 2) pre-training with Changen zheng2023scalable binary synthetic change detection data (90k pairs), and 3) pre-training with our multi-class synthetic change detection data (10k pairs). They all use SCanNet with ResNet-18 as the backbone.