Dynamics-aware Diffusion Models for Planning and Control

TL;DR

The paper tackles the challenge of generating dynamically admissible trajectories with diffusion models when system dynamics are crucial. It introduces a sequential prediction and projection mechanism that enforces system dynamics during the diffusion denoising process, aligning with the noising schedule to produce physically plausible trajectories. The framework handles both known and unknown dynamics: (a) a dynamics projection using a forward map $\mathcal{F}$ and its pseudoinverse, and (b) a data-driven Hankel-projection based on Willems' Fundamental Lemma, enabling trajectory generation from limited or no explicit dynamic knowledge. Empirical results on an LQR task and a non-convex waypoint tracking problem show substantial improvements over vanilla diffusion methods, including accurate recovery of linear-feedback trajectories and reduced trajectory and control errors, signaling practical impact for planning and control in robotics.

Abstract

This paper addresses the problem of generating dynamically admissible trajectories for control tasks using diffusion models, particularly in scenarios where the environment is complex and system dynamics are crucial for practical application. We propose a novel framework that integrates system dynamics directly into the diffusion model's denoising process through a sequential prediction and projection mechanism. This mechanism, aligned with the diffusion model's noising schedule, ensures generated trajectories are both consistent with expert demonstrations and adhere to underlying physical constraints. Notably, our approach can generate maximum likelihood trajectories and accurately recover trajectories generated by linear feedback controllers, even when explicit dynamics knowledge is unavailable. We validate the effectiveness of our method through experiments on standard control tasks and a complex non-convex optimal control problem involving waypoint tracking and collision avoidance, demonstrating its potential for efficient trajectory generation in practical applications. Our code repository is available at www.github.com/darshangm/dynamics-aware-diffusion.

Figures (3)

• Figure 1: Example of process of forward diffusion and reverse denoising to generate new trajectories. The red, green and blue points represent the initial, final and intermediate states of a trajectory $\tau_0$, respectively. The forward diffusion process progressively adds noise to the trajectory $\tau_0$ over $L$ steps, resulting in a noisy trajectory $\tau_L$. The reverse denoising process starts from a sample of $\mathcal{N}(0,I)$ that is refinedto recover a new trajectory $\tau'_0$.
• Figure 2: Comparison of average state and control error for LQR trajectory generation for the discretized double integrator with white Gaussian noise. The vanilla diffusion model, lacking dynamics awareness, exhibits high state and control errors. In contrast, our methods with known (Algorithm \ref{['algo:denoising-known-dynamics']}) and unknown (Algorithm \ref{['algo:denoising-unknown-dynamics']}) dynamics perform significantly better over the entire control horizon, demonstrating the effectiveness of incorporating dynamics into the diffusion process. Average over 100 test cases are shown.
• Figure 3: Non-convex optimal control with waypoint tracking and obstacle avoidance. (a) Example of a dataset sample illustrating a solution to the non-convex optimal control problem \ref{['eqn:complex_task']}. The solid blue line depicts the numerically computed optimal state trajectory, blue dots represent the waypoints, and pale red circles indicate obstacles. (b) Comparison of sampled state trajectories generated by different algorithms. Algorithms 1 (known dynamics) and 2 (unknown dynamics) produce smooth, dynamics-aware trajectories closely resembling the true optimal trajectory, while the vanilla diffusion model deviates significantly. (c) Average state error versus time, and (d) average control error versus time, computed over 100 test cases. Algorithms \ref{['algo:denoising-known-dynamics']} and \ref{['algo:denoising-unknown-dynamics']} exhibit significantly lower errors due to their incorporation of system dynamics, demonstrating the effectiveness of our proposed framework.

Theorems & Definitions (3)

• Lemma 1: Generation of admissible trajectories
• Theorem 2: Generation of trajectories for linear feedback control
• Remark 1