Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Constrained Synthesis with Projected Diffusion Models

Jacob K Christopher, Stephen Baek, Ferdinando Fioretto

TL;DR

The paper tackles the problem of enforcing hard constraints in diffusion-based generative models by introducing Projected Diffusion Models (PDM).PDM reframes the reverse diffusion sampling as a constrained optimization problem and enforces feasibility via iterative projections onto a constraint set during the sampling process, preserving the data distribution while satisfying constraints.The authors provide a theoretical justification for the projection-based approach, including feasibility guarantees for convex constraint sets, and validate the method experimentally across constrained materials, physics-informed motion, non-convex trajectory planning, and ODE-driven scenarios, showing constraint satisfaction with competitive fidelity.The work highlights practical trade-offs, such as computational overhead from projections, and suggests avenues for efficiency and extension to forward-process constraints and more complex multi-task constraints.

Abstract

This paper introduces an approach to endow generative diffusion processes the ability to satisfy and certify compliance with constraints and physical principles. The proposed method recast the traditional sampling process of generative diffusion models as a constrained optimization problem, steering the generated data distribution to remain within a specified region to ensure adherence to the given constraints. These capabilities are validated on applications featuring both convex and challenging, non-convex, constraints as well as ordinary differential equations, in domains spanning from synthesizing new materials with precise morphometric properties, generating physics-informed motion, optimizing paths in planning scenarios, and human motion synthesis.

Constrained Synthesis with Projected Diffusion Models

TL;DR

The paper tackles the problem of enforcing hard constraints in diffusion-based generative models by introducing Projected Diffusion Models (PDM).PDM reframes the reverse diffusion sampling as a constrained optimization problem and enforces feasibility via iterative projections onto a constraint set during the sampling process, preserving the data distribution while satisfying constraints.The authors provide a theoretical justification for the projection-based approach, including feasibility guarantees for convex constraint sets, and validate the method experimentally across constrained materials, physics-informed motion, non-convex trajectory planning, and ODE-driven scenarios, showing constraint satisfaction with competitive fidelity.The work highlights practical trade-offs, such as computational overhead from projections, and suggests avenues for efficiency and extension to forward-process constraints and more complex multi-task constraints.

Abstract

This paper introduces an approach to endow generative diffusion processes the ability to satisfy and certify compliance with constraints and physical principles. The proposed method recast the traditional sampling process of generative diffusion models as a constrained optimization problem, steering the generated data distribution to remain within a specified region to ensure adherence to the given constraints. These capabilities are validated on applications featuring both convex and challenging, non-convex, constraints as well as ordinary differential equations, in domains spanning from synthesizing new materials with precise morphometric properties, generating physics-informed motion, optimizing paths in planning scenarios, and human motion synthesis.
Paper Structure (44 sections, 3 theorems, 18 equations, 16 figures, 2 tables, 2 algorithms)

This paper contains 44 sections, 3 theorems, 18 equations, 16 figures, 2 tables, 2 algorithms.

Table of Contents

  1. Introduction
  2. Preliminaries: Diffusion models
  3. Related work and limitations
  4. Constrained generative diffusion
  5. Casting the reverse process as an optimization problem
  6. Constrained guidance through iterative projections
  7. Effectiveness of PDM: A theoretical justification
  8. Feasibility guarantees.
  9. Experiments
  10. Constrained materials (low data regimes and constraint-violating distributions)
  11. 3D human motion (dynamic motion and physical principles)
  12. Constrained trajectories ( nonconvex constraints)
  13. Physics-informed motion (ODEs and out-of-distribution constraints)
  14. Discussion and limitations
  15. Conclusions
  16. ...and 29 more sections

Key Result

Theorem 5.2

Let $\mathcal{P}_{\mathbf{C}}$ be a projection onto $\mathbf{C}$, $\bm{x}_{t}^{i}$ be the sample at time step $\bm{t}$ and iteration $\bm{i}$, and 'Error' be the cost of the projection (eq:projection). Assume $\nabla_{\bm{x}_t} \log p(\bm{x}_t)$ is convex. For any $\bm{i} \geq \Bar{I}$,

Figures (16)

  • Figure 1: Sampling steps failing to converge to feasible solutions in conditional models (red) while minimizing the constraint divergence to $0$ under PDM (blue).
  • Figure 2: Conditional diffusion model (Cond): Frequency of porosity constraint satisfaction (y-axis) within an error tolerance (x-axis) over 100 runs.
  • Figure 3: Porosity constrained microstructure visualization at varying of the imposed porosity constraint amounts (P) and FID scores.
  • Figure 4: PDM (left, FID: 0.71) and conditional (Cond) (right, FID: 0.63) generation.
  • Figure 5: Constrained trajectories synthesized by PDM on two topographies (Tp1, left and Tp2, right).
  • ...and 11 more figures

Theorems & Definitions (5)

  • Definition 5.1
  • Theorem 5.2
  • Corollary 5.3
  • Corollary 5.4
  • proof