Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Deep Generative Prior for First Order Inverse Optimization

Haoyu Yang, Kamyar Azizzadenesheli, Haoxing Ren

TL;DR

Inverse design seeks to recover inputs $a$ from observations $u^*$ under the forward map $\mathcal{F}$, but is typically ill-posed and challenging for gradient-based methods. The authors introduce Deep Generative Prior (DGP), which couples a differentiable forward surrogate $F_\theta$ with a learned generative prior $G_\phi$ and performs Langevin dynamics in the latent space to produce diverse, physically plausible inverse solutions, yielding posterior samples rather than single estimates. They provide a bound on inverse design error $\mathcal{L}(\hat{a}) \le \mathcal{L}(a^*) + L_F \epsilon_G + 2\epsilon_F$ and demonstrate strong performance across 2D Darcy flow, ill-posed Navier–Stokes, and inverse lithography tasks, achieving higher quality solutions with orders-of-magnitude speedups over MCMC baselines and robustness to out-of-distribution targets. The data-driven, surrogate-based framework offers scalable, multi-modal inverse-design capabilities when explicit physics priors are unavailable, with broad implications for engineering domains where prior distributions are unknown or difficult to encode.

Abstract

Inverse design optimization aims to infer system parameters from observed solutions, posing critical challenges across domains such as semiconductor manufacturing, structural engineering, materials science, and fluid dynamics. The lack of explicit mathematical representations in many systems complicates this process and makes the first order optimization impossible. Mainstream approaches, including generative AI and Bayesian optimization, address these challenges but have limitations. Generative AI is computationally expensive, while Bayesian optimization, relying on surrogate models, suffers from scalability, sensitivity to priors, and noise issues, often leading to suboptimal solutions. This paper introduces Deep Physics Prior (DPP), a novel method enabling first-order gradient-based inverse optimization with surrogate machine learning models. By leveraging pretrained auxiliary Neural Operators, DPP enforces prior distribution constraints to ensure robust and meaningful solutions. This approach is particularly effective when prior data and observation distributions are unknown.

Deep Generative Prior for First Order Inverse Optimization

TL;DR

Inverse design seeks to recover inputs from observations under the forward map , but is typically ill-posed and challenging for gradient-based methods. The authors introduce Deep Generative Prior (DGP), which couples a differentiable forward surrogate with a learned generative prior and performs Langevin dynamics in the latent space to produce diverse, physically plausible inverse solutions, yielding posterior samples rather than single estimates. They provide a bound on inverse design error and demonstrate strong performance across 2D Darcy flow, ill-posed Navier–Stokes, and inverse lithography tasks, achieving higher quality solutions with orders-of-magnitude speedups over MCMC baselines and robustness to out-of-distribution targets. The data-driven, surrogate-based framework offers scalable, multi-modal inverse-design capabilities when explicit physics priors are unavailable, with broad implications for engineering domains where prior distributions are unknown or difficult to encode.

Abstract

Inverse design optimization aims to infer system parameters from observed solutions, posing critical challenges across domains such as semiconductor manufacturing, structural engineering, materials science, and fluid dynamics. The lack of explicit mathematical representations in many systems complicates this process and makes the first order optimization impossible. Mainstream approaches, including generative AI and Bayesian optimization, address these challenges but have limitations. Generative AI is computationally expensive, while Bayesian optimization, relying on surrogate models, suffers from scalability, sensitivity to priors, and noise issues, often leading to suboptimal solutions. This paper introduces Deep Physics Prior (DPP), a novel method enabling first-order gradient-based inverse optimization with surrogate machine learning models. By leveraging pretrained auxiliary Neural Operators, DPP enforces prior distribution constraints to ensure robust and meaningful solutions. This approach is particularly effective when prior data and observation distributions are unknown.
Paper Structure (47 sections, 1 theorem, 21 equations, 6 figures, 5 tables)

This paper contains 47 sections, 1 theorem, 21 equations, 6 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Fourier Neural Operator (FNO).
  4. Deep Generative Models As Inverse Operator.
  5. Adversarial Sampling.
  6. Physics-Informed Neural Networks (PINNs).
  7. Connections to Deep Generative Prior (DGP).
  8. The Methodology
  9. Problem Formulation
  10. Forward Operator.
  11. Inverse Design.
  12. Adversarial Behavior of $F_\theta$ in Inverse Design
  13. Deep Generative Prior
  14. Posterior Sampling.
  15. Discussion
  16. ...and 32 more sections

Key Result

Lemma 1

Assume: (1) $G_\phi$ can approximate $a^\ast$ within error $\epsilon_G$, i.e., $\|G_\phi(q^\ast)-a^\ast\|\le \epsilon_G$ for some $q^\ast$; (2) the surrogate $F_\theta$ is $L_F$-Lipschitz; (3) the surrogate approximates the true forward operator within error $\epsilon_F$ on the generator range, i.e.

Figures (6)

  • Figure 1: Comparison of inverse design optimization schemes. (a) Data-driven generative model mapping objectives to parameters. (b) MCMC sampling using a pretrained surrogate. (c) First-order optimization of system parameters constrained by a deep generative prior (DGP).
  • Figure 2: Visualization of inverse Darcy flow with exponentiated permeability (top) and clipped permeability (bottom). Left column: ground-truth reference flow pressure and permeability. Right columns: baseline methods and our deep generative prior with LD posterior sampling.
  • Figure 3: Visualization of inverse Navier-Stokes flow on vorticity field. Left column: ground-truth reference vorticity at time step 0 ($w(x,0)$) and time step T ($w(x,T)$). Right columns: baseline methods and our deep generative prior with LD posterior sampling.
  • Figure 4: Visualization of posterior sampling of five inverse solutions under ill-posed settings. Top: sampled $w(x,0)$; Bottom: solver derived $w(x, T)$.
  • Figure 5: Visualization of inverse lithography solutions. (a) Snippet of the chip design. (b) The suboptimal mask of (a) in the dataset using numerical solver. (c) The wafer image of (b). (d) The optimized mask using LD. (e) The wafer image of (d).
  • ...and 1 more figures

Theorems & Definitions (2)

  • Lemma 1
  • proof