Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Diffusion Models with Double Guidance: Generate with aggregated datasets

Yanfeng Yang, Kenji Fukumizu

TL;DR

The paper tackles the challenge of learning conditional diffusion models from aggregated datasets with block-wise missing conditions. It introduces two double-guidance methods, DMDG and DMHG, which decompose the conditional score into an unconditional term plus two conditioning-guidance terms, enabling joint conditioning on $P_{X_0|C_1,C_2}$ without requiring joint annotations. Theoretical analysis bounds the density discrepancy due to approximations and highlights factors like noise variances, gradient magnitudes, and Tweedie projections that govern performance. Empirically, the methods outperform independent-guidance baselines across synthetic, molecular design, and image inpainting tasks, with CFG-based variants often delivering stronger alignment under complex conditioning. This work broadens the applicability of conditional diffusion models to heterogeneous, partially labeled data, offering practical gains for drug design and computer vision tasks.

Abstract

Creating large-scale datasets for training high-performance generative models is often prohibitively expensive, especially when associated attributes or annotations must be provided. As a result, merging existing datasets has become a common strategy. However, the sets of attributes across datasets are often inconsistent, and their naive concatenation typically leads to block-wise missing conditions. This presents a significant challenge for conditional generative modeling when the multiple attributes are used jointly as conditions, thereby limiting the model's controllability and applicability. To address this issue, we propose a novel generative approach, Diffusion Model with Double Guidance, which enables precise conditional generation even when no training samples contain all conditions simultaneously. Our method maintains rigorous control over multiple conditions without requiring joint annotations. We demonstrate its effectiveness in molecular and image generation tasks, where it outperforms existing baselines both in alignment with target conditional distributions and in controllability under missing condition settings.

Diffusion Models with Double Guidance: Generate with aggregated datasets

TL;DR

The paper tackles the challenge of learning conditional diffusion models from aggregated datasets with block-wise missing conditions. It introduces two double-guidance methods, DMDG and DMHG, which decompose the conditional score into an unconditional term plus two conditioning-guidance terms, enabling joint conditioning on without requiring joint annotations. Theoretical analysis bounds the density discrepancy due to approximations and highlights factors like noise variances, gradient magnitudes, and Tweedie projections that govern performance. Empirically, the methods outperform independent-guidance baselines across synthetic, molecular design, and image inpainting tasks, with CFG-based variants often delivering stronger alignment under complex conditioning. This work broadens the applicability of conditional diffusion models to heterogeneous, partially labeled data, offering practical gains for drug design and computer vision tasks.

Abstract

Creating large-scale datasets for training high-performance generative models is often prohibitively expensive, especially when associated attributes or annotations must be provided. As a result, merging existing datasets has become a common strategy. However, the sets of attributes across datasets are often inconsistent, and their naive concatenation typically leads to block-wise missing conditions. This presents a significant challenge for conditional generative modeling when the multiple attributes are used jointly as conditions, thereby limiting the model's controllability and applicability. To address this issue, we propose a novel generative approach, Diffusion Model with Double Guidance, which enables precise conditional generation even when no training samples contain all conditions simultaneously. Our method maintains rigorous control over multiple conditions without requiring joint annotations. We demonstrate its effectiveness in molecular and image generation tasks, where it outperforms existing baselines both in alignment with target conditional distributions and in controllability under missing condition settings.
Paper Structure (34 sections, 4 theorems, 42 equations, 9 figures, 7 tables, 1 algorithm)

This paper contains 34 sections, 4 theorems, 42 equations, 9 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Problem setting and preliminaries
  3. Problem setting
  4. EDM framework karras_edm
  5. Classifier Guidance (CG) and Diffusion Posterior Sampling (DPS)
  6. Classifier-Free Guidance (CFG)
  7. Methods of Double Guidance
  8. Diffusion model with double guidance (DMDG)
  9. Diffusion Model with Hybrid Guidance (DMHG)
  10. Existing method: Diffusion Model with Independent Double Guidance (DMIDG)
  11. Theoretical Analysis
  12. Simulation Study with Synthetic Data
  13. Experimetns with Real-world Data
  14. Molecule generation
  15. Image inpainting with DMHG
  16. ...and 19 more sections

Key Result

Theorem 1

Assume that $p(C_1|X_t)$ and $p(C_2|X_t,C_1)$ are the densities of $C_1$ given $X_t$, and $C_2$ given $(X_t,C_1)$, respectively, and that $p(C_1|X_{0|t})$ and $p(C_2|X_{0|t,C_1})$ are densities of the normal distribution defined in (relationship_nerual_estimator). Let $M$ be the maximum $\ell_2$-nor

Figures (9)

  • Figure 1: Comparison of score functions when $t=1$. In the left panel, both the double guidance and independent guidance approaches accurately replicate the conditional score function $\nabla_{X_t} \log p_t(X_t|C_1,C_2)$. In contrast, as illustrated in the right panel, the score function guided by independent guidance deviates significantly from $\nabla_{X_t} \log p_t(X_t|C_1,C_2)$ , whereas DMDG continues to align with the target score function.
  • Figure 2: Success rate of Tasks 1 & 2 (\ref{['eq_molecular_task_12']}) and $W_2$ distances to GEOM and ZINC250k dataset.
  • Figure 3: Images generated by DMHG, DMIHG and DPS. The images generated by DMHG exhibit both high visual quality and strong diversity.All parameters are tuned to minimize LPIPS.
  • Figure 4: Left panel: Conditional independence relationships among $X_0, X_t,C_1$ and $C_2$: $X_t, C_1,C_2$ are mutually independent given $X_0$. While $C_1 \perp \! \! \! \perp C_2 |X_{t}$ and $C_1 \perp \! \! \! \perp C_2 |X_{0|t}$ is not true. Right panel: The behavior of conditional mutual informations (CMI) over the scale of $t$ in forward process (\ref{['eq_forward_process']}).
  • Figure 5: The LPIPS, PSNR, accuracy, MSE and their standard deviations of inpainting task. $\lambda_1$ is set to 1.
  • ...and 4 more figures

Theorems & Definitions (6)

  • Theorem 1
  • Remark 1
  • Lemma 2
  • Lemma 3
  • Theorem 4
  • Remark 2