Physics-Informed Diffusion Models

TL;DR

This work introduces physics-informed diffusion models (PIDMs) that embed first-principles PDE constraints into the training of denoising diffusion models, via a residual likelihood term that enforces physical laws alongside data fidelity. By combining PDE residuals with standard data likelihood, PIDMs achieve substantial improvements in PDE fulfillment while preserving generative diversity, demonstrated on Darcy flow and topology optimization tasks. The approach yields up to two orders of magnitude reduction in PDE residuals and acts as a regularizer against overfitting, all without altering inference or requiring additional surrogate models. The framework is broadly applicable to equality and inequality constraints and promises broader impact for physics-constrained generative modeling in scientific machine learning.

Abstract

Generative models such as denoising diffusion models are quickly advancing their ability to approximate highly complex data distributions. They are also increasingly leveraged in scientific machine learning, where samples from the implied data distribution are expected to adhere to specific governing equations. We present a framework that unifies generative modeling and partial differential equation fulfillment by introducing a first-principle-based loss term that enforces generated samples to fulfill the underlying physical constraints. Our approach reduces the residual error by up to two orders of magnitude compared to previous work in a fluid flow case study and outperforms task-specific frameworks in relevant metrics for structural topology optimization. We also present numerical evidence that our extended training objective acts as a natural regularization mechanism against overfitting. Our framework is simple to implement and versatile in its applicability for imposing equality and inequality constraints as well as auxiliary optimization objectives.

Figures (9)

• Figure 1: An approximation $\boldsymbol{x}_0^*$ of the clean signal for residual evaluation can be obtained at any denoising timestep $t$ by directly considering the estimated expectation $\mathbb{E}[\boldsymbol{x}_0\vert\boldsymbol{x}_t]$ or by actual (accelerated DDIM Song2020) sampling. We tighten the variance of the virtual likelihood as $t\to0$.
• Figure 2: Evaluation of the residual error (a) and test data loss (b) during training. In (a), we generate 16 samples every 10k training iterations and plot the average (solid lines) and individual (dots) residual errors for the standard diffusion model ('Diffusion'), the physics-guided model ('PG-Diffusion') Shu2023, CoCoGen Jacobsen2023, and the proposed PIDM using either mean or sample estimation ('PIDM-ME' and 'PIDM-SE', respectively). Note that for CoCoGen not all samples converged, so that we excluded the non-converged data from the indicated average. In (b), we plot the data loss evaluated on a test set for the proposed PIDM variants and those frameworks that differ from ours during training.
• Figure 3: Generated permeability and pressure fields as well as the corresponding residual error from diffusion models trained on the Darcy flow dataset, where (a) is sampled from a standard diffusion model and (b) from our proposed PIDM with mean estimation. Additional samples are shown in Appendix \ref{['app:add_darcy_samples']}.
• Figure 4: Generated designs, including the compliance error CE and volume $\bar{\rho}$, and the residual error (based on the displacement fields, not shown) from diffusion models trained on the SIMP dataset and the corresponding SIMP design, including the compliance C and volume $V_{\text{max}}$. We plot a sample (a) from a standard diffusion model and (b) from our proposed PIDM. All samples are conditioned on the out-of-distribution test set. In the SIMP design, we indicate the applied load by a blue dot, and the given boundary conditions in red.
• Figure 5: (Mean estimation) Evaluation of the average residual error of 100 generated samples during training (a, averaged over 10 training runs with 25/75%-quantiles using different seeds) and 100 generated samples after training (b, from representative models) in four different settings. We consider a diffusion model trained with the standard (data-driven) objective (i), our proposed PIDM (ii), a model trained solely on the residual loss term (iii), and again our proposed PIDM but with data sampled from an uninformative Gaussian prior (iv). The residual during training is evaluated via mean estimation, i.e., $\boldsymbol{x}_0^*=\mathbb{E}[\boldsymbol{x}_0\vert\boldsymbol{x}_t]$. In (b), we indicate the unit circle to which all samples should be constrained. Colors in (b) match those in (a).

Theorems & Definitions (3)

• Proposition 1
• proof
• Remark