FDS: Feedback-guided Domain Synthesis with Multi-Source Conditional Diffusion Models for Domain Generalization

TL;DR

The paper tackles domain generalization by introducing Feedback-guided Domain Synthesis (FDS), a diffusion-based framework that trains a single conditional diffusion model on multiple source domains and synthesizes diverse pseudo-domains through domain interpolation. Two mixing strategies—noise-level and condition-level interpolation—generate inter-domain samples, which are then filtered by an entropy-based criterion to select informative examples for training. Empirical results on DG benchmarks (PACS, VLCS, OfficeHome) show consistent improvements over ERM and competitive gains versus SOTA methods, with ablations highlighting the critical role of the filtering step in boosting OOD generalization. The approach also demonstrates regularization benefits in in-domain settings and provides intuitive visual evidence (t-SNE, inter-domain transitions) of enhanced domain coverage, suggesting practical impact for robust vision systems in the face of distribution shifts.

Abstract

Domain Generalization techniques aim to enhance model robustness by simulating novel data distributions during training, typically through various augmentation or stylization strategies. However, these methods frequently suffer from limited control over the diversity of generated images and lack assurance that these images span distinct distributions. To address these challenges, we propose FDS, Feedback-guided Domain Synthesis, a novel strategy that employs diffusion models to synthesize novel, pseudo-domains by training a single model on all source domains and performing domain mixing based on learned features. By incorporating images that pose classification challenges to models trained on original samples, alongside the original dataset, we ensure the generation of a training set that spans a broad distribution spectrum. Our comprehensive evaluations demonstrate that this methodology sets new benchmarks in domain generalization performance across a range of challenging datasets, effectively managing diverse types of domain shifts. The code can be found at: \url{https://github.com/Mehrdad-Noori/FDS.git}.

Figures (16)

• Figure 1: Generating new, pseudo-domains with FDS: Comprehensive distribution coverage from domain $D_1$ to $D_2$.
• Figure 2: Overview of the proposed architecture for FDS. (top) Multi-source training of diffusion model conditioned on class and domain of the training images. (bottom) Generating novel pseudo-domain using the proposed interpolation and filtering mechanism of FDS.
• Figure 3: Impact of varying scales of sample size $N_L$ relative to the average number of images per class on PACS dataset.
• Figure 4: Impact of using Random Selection vs. Proposed Filtering Strategy of FDS on PACS accuracy (%).
• Figure 5: t-SNE plots showcasing the original "giraffe" class samples for the "Art", "Photo" and "Sketch" source domains of the PACS dataset, as well as the data generated with FDS.