Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Constrained Generative Modeling with Manually Bridged Diffusion Models

Saeid Naderiparizi, Xiaoxuan Liang, Berend Zwartsenberg, Frank Wood

TL;DR

Constrained Generative Modeling with Manually Bridged Diffusion Models introduces MBM, a diffusion-based framework for constrained sampling defined on a set $\Omega$ that uses manually constructed bridges to steer samples toward feasibility. It formalizes an $\Omega$-distance function $\ell^\Omega$ and a time-dependent bridge $\mathbf{b}^\Omega(\mathbf{x}; t) = -\gamma(t) \nabla_x \ell^\Omega(\mathbf{x}; t)$ and shows how multiple bridges can be added to enforce intersections $\cap_i \Omega_i$, while preserving a standard diffusion objective. The paper analyzes three architectures—$\text{C-arch}$, $\text{DB-arch}$, and $\text{MBM-arch}$—and provides empirical evidence that $\text{MBM-arch}$ yields near-zero constraint violations with competitive or superior training efficiency compared to diffusion-bridge baselines, on both checkerboard and traffic-scene tasks. The work emphasizes applicability to safety-critical planning, provides practical training strategies, and outlines theoretical directions for formal guarantees and extensions to trajectory spaces.

Abstract

In this paper we describe a novel framework for diffusion-based generative modeling on constrained spaces. In particular, we introduce manual bridges, a framework that expands the kinds of constraints that can be practically used to form so-called diffusion bridges. We develop a mechanism for combining multiple such constraints so that the resulting multiply-constrained model remains a manual bridge that respects all constraints. We also develop a mechanism for training a diffusion model that respects such multiple constraints while also adapting it to match a data distribution. We develop and extend theory demonstrating the mathematical validity of our mechanisms. Additionally, we demonstrate our mechanism in constrained generative modeling tasks, highlighting a particular high-value application in modeling trajectory initializations for path planning and control in autonomous vehicles.

Constrained Generative Modeling with Manually Bridged Diffusion Models

TL;DR

Constrained Generative Modeling with Manually Bridged Diffusion Models introduces MBM, a diffusion-based framework for constrained sampling defined on a set that uses manually constructed bridges to steer samples toward feasibility. It formalizes an -distance function and a time-dependent bridge and shows how multiple bridges can be added to enforce intersections , while preserving a standard diffusion objective. The paper analyzes three architectures—, , and —and provides empirical evidence that yields near-zero constraint violations with competitive or superior training efficiency compared to diffusion-bridge baselines, on both checkerboard and traffic-scene tasks. The work emphasizes applicability to safety-critical planning, provides practical training strategies, and outlines theoretical directions for formal guarantees and extensions to trajectory spaces.

Abstract

In this paper we describe a novel framework for diffusion-based generative modeling on constrained spaces. In particular, we introduce manual bridges, a framework that expands the kinds of constraints that can be practically used to form so-called diffusion bridges. We develop a mechanism for combining multiple such constraints so that the resulting multiply-constrained model remains a manual bridge that respects all constraints. We also develop a mechanism for training a diffusion model that respects such multiple constraints while also adapting it to match a data distribution. We develop and extend theory demonstrating the mathematical validity of our mechanisms. Additionally, we demonstrate our mechanism in constrained generative modeling tasks, highlighting a particular high-value application in modeling trajectory initializations for path planning and control in autonomous vehicles.

Paper Structure

This paper contains 49 sections, 5 theorems, 29 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Background
  4. Diffusion Models
  5. Constrained Generative Modeling
  6. Diffusion Bridges
  7. Methodology
  8. Manual Bridges
  9. Connection to diffusion bridges
  10. Architecture
  11. Experiments
  12. Checkerboard Constraint Experiment
  13. Setup
  14. Diffusion bridge baseline
  15. Manual bridges
  16. ...and 34 more sections

Key Result

Proposition 1

Let $s_\theta({\mathbf{x}}, t)$ be a score function corresponding to a density $p_\theta({\mathbf{x}}, t)$. If $s_\theta({\mathbf{x}}, t)$ is continuous in $t$ and $p_\theta({\mathbf{x}}, t)$ is finite for ${\mathbf{x}} \notin \Omega$, then the manually bridged model in def:manually-bridged-model re

Figures (6)

  • Figure 1: MBM applied to traffic scene generation. The goal is to place vehicles on a given bird's-eye view image of a map; indicated here as the "light" region of an underlying aerial image. The model output is the set of "cars" (infraction-free cars are green; cars involved in infractions are yellow). The top row shows samples from different models given the same map. The standard diffusion sample (\ref{['fig:banner-baseline']}) contains a collision infraction. The rest of the top row shows different architectural mechanisms to avoid infractions. Both conditional diffusion (\ref{['fig:banner-conditional']}) and (\ref{['fig:banner-sde']}) are not realistic: they both distort the distribution, albeit in different ways, this effect being more apparent in (\ref{['fig:banner-sde']}). A sample from MBM in (\ref{['fig:banner-sde-inp']}) shows no infractions while remaining realistic. The second row shows samples from MBM on additional maps.
  • Figure 2: Score function architectural variants considered. The latter three use the same "manual bridge," with the last notably including an additional path for the bridge function gradient not previously considered in the literature. Each diagram shows a single denoising step. In these diagrams the input $t$ to the model is omitted for conciseness.
  • Figure 3: Visualization of the checkerboard constraint experiment results. The problem is constrained on a checkerboard pattern and the data has a uniform distribution over triangles within the checkerboard, shown in (\ref{['fig:toy-vis:data']}). Invalid samples are shown in brown. (\ref{['fig:toy-vis:prior-analytic']}, \ref{['fig:toy-vis:prior-manual']}) show the diffusion bridge and manually bridged models without a trained diffusion model. This is effectively the prior distributions in these models. As shown on the second row, bridged models do not produce invalid samples. Further, the manually bridged model gives comparable samples to the diffusion bridges liu2023learning. Finally, incorporating the bridges in our proposed way (labelled with MBM) even improves the diffusion bridge models.
  • Figure 4: Checkerboard constraint experiment results. Solid, dashed and dotted lines respectively correspond to manual bridge, diffusion bridge and baseline models. Different colors represent different mechanisms of incorporating bridges as shown in \ref{['fig:implementation-diagrams']}. The shaded area shows standard deviation of the metrics over three models trained with different random seeds.
  • Figure 5: While standard diffusion models struggle to achieve zero infraction, incorporating diffusion bridges ensures constraint satisfaction.
  • ...and 1 more figures

Theorems & Definitions (11)

  • Definition 1: $\Omega$-distance function
  • Definition 2: Manually bridged model
  • Proposition 1
  • Proposition 2: Combining Manual Bridges
  • proof
  • Lemma 1
  • proof
  • Proposition 3
  • proof
  • Lemma 2
  • ...and 1 more