Training-Free Constrained Generation With Stable Diffusion Models

TL;DR

This work addresses the challenge of producing diffusion-generated outputs that strictly satisfy domain-specific constraints without retraining. It introduces a training-free latent-space correction framework that leverages ambient-space constraint evaluation through a differentiable decoder and applies Proximal Langevin Dynamics to enforce feasibility at inference time. The method provides convergence and fidelity guarantees, extends to non-convex and black-box constraints via surrogate and DPO-based strategies, and demonstrates strong results in microstructure porosity control, metamaterial stress-strain design, and copyright-safe content generation. The approach maintains high sample quality while offering rigorous constraint adherence, promising safer and more practically deployable diffusion-based systems.

Abstract

Stable diffusion models represent the state-of-the-art in data synthesis across diverse domains and hold transformative potential for applications in science and engineering, e.g., by facilitating the discovery of novel solutions and simulating systems that are computationally intractable to model explicitly. While there is increasing effort to incorporate physics-based constraints into generative models, existing techniques are either limited in their applicability to latent diffusion frameworks or lack the capability to strictly enforce domain-specific constraints. To address this limitation this paper proposes a novel integration of stable diffusion models with constrained optimization frameworks, enabling the generation of outputs satisfying stringent physical and functional requirements. The effectiveness of this approach is demonstrated through material design experiments requiring adherence to precise morphometric properties, challenging inverse design tasks involving the generation of materials inducing specific stress-strain responses, and copyright-constrained content generation tasks. All code has been released at https://github.com/RAISELab-atUVA/Constrained-Stable-Diffusion.

Figures (10)

• Figure 1: Comparison of model performance in terms of FID score and constraint satisfaction (percentage of samples that does not satisfy the target porosity with a margin of 10$\%$).
• Figure 2: Distribution of void diameters in the training set (Ground) and in data generated by Conditional diffusion model and Latent Constrained Diffusion models.
• Figure 3: Compare MSE w.r.t. target stress-strain response and rejection rate of physically inconsistent shapes.
• Figure 4: Successive steps of DPO. The sample is iteratively improved and the stress-strain curve aligns with the target. Structural analysis shows progressive deformation under controlled compression.
• Figure 5: Left: Denoising process of Cond vs. Latent (Ours). Out method drives the denoising toward a copyright-safe image. Top-right: Showing projection from original (O) to projected (P) in the PCA-2 space. Bottom-right: Constraint satisfaction and FID scores.

Theorems & Definitions (11)

• Theorem 4.1: Convergence to the Constraint Set
• Theorem 4.2: Training Distribution Fidelity
• Theorem G.1
• proof
• Lemma H.1
• proof : Proof of non-expansiveness of the projection operator
• Theorem H.2
• proof
• proof
• Lemma H.4