Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

From Prompts to Deployment: Auto-Curated Domain-Specific Dataset Generation via Diffusion Models

Dongsik Yoon, Jongeun Kim

TL;DR

The paper tackles deployment distribution shift by enabling domain-specific dataset creation through diffusion models. It introduces a three-stage pipeline: Stage 1 synthesizes target objects into domain backgrounds via inpainting with multi-modal screening using $S_{det}$, $S_{aes}$, $IoU(B_{det}, M)$, and $S_{vlm}$, Stage 2 trains a DINOv2+ConvNeXt preference classifier to encode user criteria, and Stage 3 constructs the final dataset by combining these components. On an elevator CCTV domain with dogs, the approach achieves high-quality synthetic data, with the proposed classifier attaining the best F1-score of $0.8427$ among tested backbones. The work reduces data collection costs and enables deployment-ready datasets while noting limitations such as cross-domain generalization and dependency on pre-trained detectors.

Abstract

In this paper, we present an automated pipeline for generating domain-specific synthetic datasets with diffusion models, addressing the distribution shift between pre-trained models and real-world deployment environments. Our three-stage framework first synthesizes target objects within domain-specific backgrounds through controlled inpainting. The generated outputs are then validated via a multi-modal assessment that integrates object detection, aesthetic scoring, and vision-language alignment. Finally, a user-preference classifier is employed to capture subjective selection criteria. This pipeline enables the efficient construction of high-quality, deployable datasets while reducing reliance on extensive real-world data collection.

From Prompts to Deployment: Auto-Curated Domain-Specific Dataset Generation via Diffusion Models

TL;DR

The paper tackles deployment distribution shift by enabling domain-specific dataset creation through diffusion models. It introduces a three-stage pipeline: Stage 1 synthesizes target objects into domain backgrounds via inpainting with multi-modal screening using , , , and , Stage 2 trains a DINOv2+ConvNeXt preference classifier to encode user criteria, and Stage 3 constructs the final dataset by combining these components. On an elevator CCTV domain with dogs, the approach achieves high-quality synthetic data, with the proposed classifier attaining the best F1-score of among tested backbones. The work reduces data collection costs and enables deployment-ready datasets while noting limitations such as cross-domain generalization and dependency on pre-trained detectors.

Abstract

In this paper, we present an automated pipeline for generating domain-specific synthetic datasets with diffusion models, addressing the distribution shift between pre-trained models and real-world deployment environments. Our three-stage framework first synthesizes target objects within domain-specific backgrounds through controlled inpainting. The generated outputs are then validated via a multi-modal assessment that integrates object detection, aesthetic scoring, and vision-language alignment. Finally, a user-preference classifier is employed to capture subjective selection criteria. This pipeline enables the efficient construction of high-quality, deployable datasets while reducing reliance on extensive real-world data collection.
Paper Structure (9 sections, 4 figures, 1 table)

This paper contains 9 sections, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Method
  3. Stage 1: Object Synthesis and Initial Screening
  4. Stage 2: Preference Classification
  5. Stage 3: Final Dataset Construction
  6. Experiments
  7. Implementation Details
  8. Backbone Details
  9. Conclusion

Figures (4)

  • Figure 1: Overview of the proposed three-stage, diffusion-based dataset generation and auto-curation pipeline.
  • Figure 2: Negative images that failed object synthesis in Stage 1 due to low detection and aesthetics scores.
  • Figure 3: Discarded images rejected in Stage 2 due to poor viewpoint/pose from annotators despite successful object synthesis.
  • Figure 4: Success image that passes all three stages, demonstrating target object synthesis within a domain-specific background.