Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Optimizing Diffusion Models for Joint Trajectory Prediction and Controllable Generation

Yixiao Wang, Chen Tang, Lingfeng Sun, Simone Rossi, Yichen Xie, Chensheng Peng, Thomas Hannagan, Stefano Sabatini, Nicola Poerio, Masayoshi Tomizuka, Wei Zhan

TL;DR

Experimental validation on the large-scale Argoverse 2 dataset demonstrates the approach's superior performance, offering a viable solution for computationally efficient, high-quality joint trajectory prediction and controllable generation for autonomous driving.

Abstract

Diffusion models are promising for joint trajectory prediction and controllable generation in autonomous driving, but they face challenges of inefficient inference steps and high computational demands. To tackle these challenges, we introduce Optimal Gaussian Diffusion (OGD) and Estimated Clean Manifold (ECM) Guidance. OGD optimizes the prior distribution for a small diffusion time $T$ and starts the reverse diffusion process from it. ECM directly injects guidance gradients to the estimated clean manifold, eliminating extensive gradient backpropagation throughout the network. Our methodology streamlines the generative process, enabling practical applications with reduced computational overhead. Experimental validation on the large-scale Argoverse 2 dataset demonstrates our approach's superior performance, offering a viable solution for computationally efficient, high-quality joint trajectory prediction and controllable generation for autonomous driving. Our project webpage is at https://yixiaowang7.github.io/OptTrajDiff_Page/.

Optimizing Diffusion Models for Joint Trajectory Prediction and Controllable Generation

TL;DR

Experimental validation on the large-scale Argoverse 2 dataset demonstrates the approach's superior performance, offering a viable solution for computationally efficient, high-quality joint trajectory prediction and controllable generation for autonomous driving.

Abstract

Diffusion models are promising for joint trajectory prediction and controllable generation in autonomous driving, but they face challenges of inefficient inference steps and high computational demands. To tackle these challenges, we introduce Optimal Gaussian Diffusion (OGD) and Estimated Clean Manifold (ECM) Guidance. OGD optimizes the prior distribution for a small diffusion time and starts the reverse diffusion process from it. ECM directly injects guidance gradients to the estimated clean manifold, eliminating extensive gradient backpropagation throughout the network. Our methodology streamlines the generative process, enabling practical applications with reduced computational overhead. Experimental validation on the large-scale Argoverse 2 dataset demonstrates our approach's superior performance, offering a viable solution for computationally efficient, high-quality joint trajectory prediction and controllable generation for autonomous driving. Our project webpage is at https://yixiaowang7.github.io/OptTrajDiff_Page/.
Paper Structure (29 sections, 2 theorems, 34 equations, 13 figures, 7 tables, 1 algorithm)

This paper contains 29 sections, 2 theorems, 34 equations, 13 figures, 7 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Preliminaries
  4. Diffusion Models
  5. Guided Sampling
  6. Trajectory Prediction and Controllable Generation
  7. Methodology
  8. Optimal Gaussian Diffusion
  9. Estimated Clean Manifold Guidance with Reference
  10. Estimated Clean Manifold Guidance.
  11. Reference Joint Trajectories.
  12. Experiments
  13. Experimental setup
  14. Joint trajectory prediction
  15. Controllable Generation
  16. ...and 14 more sections

Key Result

proposition thmcounterproposition

Denote ${{\boldsymbol{{\mu}}}}_d$ and ${{\boldsymbol{{\Sigma}}}}_d$ as the mean and variance of $p_{data}$. Denote ${{\boldsymbol{{\Sigma}}}}^*(i,j)$ and ${{\boldsymbol{{\Sigma}}}}_p^*(i,j)$ as the element at $i$th row and $j$th column of matrix ${{\boldsymbol{{\Sigma}}}}^*$ and ${{\boldsymbol{{\Sig

Figures (13)

  • Figure 1: Overview of Optimal Gaussian Diffusion (OGD) and Estimated Clean Manifold Guidance (ECM). (a) OGD uses the mean and variance of the data distribution to calculate the optimal prior distribution at a small $T$. It can largely reduce the diffusion time compared with vanilla diffusion. (b) ECM directly injects the gradient of guidance into the clean data manifold to mitigate computational complexity.
  • Figure 2: Two challenges with multi-peak function optimization: 1) Gradients may lead to suboptimal local optima (left); 2) There exist regions with low likelihood but high guidance cost uncertainty, leading to instability (right). Our approach can bypass the lengthy paths between peaks, search for better optima, and avoid uncertain areas.
  • Figure 3: Evaluation of Optimal Gaussian Diffusion and vanilla diffusion over reverse steps $T$.
  • Figure 4: Evaluation on joint trajectory prediction task. For each metric, the best result is in bold and the second best result is underlined. $T=70$ is the best $T$ from \ref{['fig: ogd over T']}. $T=40$ is the minimal diffusion time when OGD outperforms $\text{VD}_{500}$ on all metrics.
  • Figure 5: Evaluation on controllable generation: route set U and Deceleration. Magenta diamonds represent goal points. In the first (second) row, goal points are set at the fork lane (right lane). NNM zhong2023guided and SF rempe2023tracejiang2023motiondiffuser struggle to drag samples from one modal to another. Our methods can achieve better guidance effectiveness and realism.
  • ...and 8 more figures

Theorems & Definitions (3)

  • proposition thmcounterproposition
  • corollary thmcountercorollary
  • proof