Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Denoising Diffusion-Based Control of Nonlinear Systems

Karthik Elamvazhuthi, Darshan Gadginmath, Fabio Pasqualetti

TL;DR

The paper reformulates nonlinear feedback control as density control by leveraging denoising diffusion probabilistic models (DDPMs). By treating control as the reverse diffusion that tracks a forward diffusion from the target density, the authors establish exact density tracking for driftless, control-affine systems under Chow-Rashevsky controllability, and provide a practical learning framework using KL-divergence minimization and neural approximators for the score. They develop a kernel-density based KL estimator for finite-sample training and demonstrate the method on a 5D bilinear system, a unicycle, and Husky robots in PyBullet, showing performance improves with more samples and measurement instances. The approach enables planning and control in non-convex environments, bridging generative modeling and nonlinear control with theoretical guarantees and empirical validation on high-dimensional systems.

Abstract

We propose a novel approach based on Denoising Diffusion Probabilistic Models (DDPMs) to control nonlinear dynamical systems. DDPMs are the state-of-art of generative models that have achieved success in a wide variety of sampling tasks. In our framework, we pose the feedback control problem as a generative task of drawing samples from a target set under control system constraints. The forward process of DDPMs constructs trajectories originating from a target set by adding noise. We learn to control a dynamical system in reverse such that the terminal state belongs to the target set. For control-affine systems without drift, we prove that the control system can exactly track the trajectory of the forward process in reverse, whenever the the Lie bracket based condition for controllability holds. We numerically study our approach on various nonlinear systems and verify our theoretical results. We also conduct numerical experiments for cases beyond our theoretical results on a physics-engine.

Denoising Diffusion-Based Control of Nonlinear Systems

TL;DR

The paper reformulates nonlinear feedback control as density control by leveraging denoising diffusion probabilistic models (DDPMs). By treating control as the reverse diffusion that tracks a forward diffusion from the target density, the authors establish exact density tracking for driftless, control-affine systems under Chow-Rashevsky controllability, and provide a practical learning framework using KL-divergence minimization and neural approximators for the score. They develop a kernel-density based KL estimator for finite-sample training and demonstrate the method on a 5D bilinear system, a unicycle, and Husky robots in PyBullet, showing performance improves with more samples and measurement instances. The approach enables planning and control in non-convex environments, bridging generative modeling and nonlinear control with theoretical guarantees and empirical validation on high-dimensional systems.

Abstract

We propose a novel approach based on Denoising Diffusion Probabilistic Models (DDPMs) to control nonlinear dynamical systems. DDPMs are the state-of-art of generative models that have achieved success in a wide variety of sampling tasks. In our framework, we pose the feedback control problem as a generative task of drawing samples from a target set under control system constraints. The forward process of DDPMs constructs trajectories originating from a target set by adding noise. We learn to control a dynamical system in reverse such that the terminal state belongs to the target set. For control-affine systems without drift, we prove that the control system can exactly track the trajectory of the forward process in reverse, whenever the the Lie bracket based condition for controllability holds. We numerically study our approach on various nonlinear systems and verify our theoretical results. We also conduct numerical experiments for cases beyond our theoretical results on a physics-engine.
Paper Structure (16 sections, 5 theorems, 34 equations, 5 figures, 1 algorithm)

This paper contains 16 sections, 5 theorems, 34 equations, 5 figures, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related work
  3. Contributions
  4. Problem setting and preliminary notions
  5. Denoising Diffusion Probabilistic model
  6. DDPM Control Problem
  7. Algorithms for DDPM-based control
  8. Theoretical Analysis and Results
  9. Numerical Studies
  10. KL-divergence approximation
  11. Experiments on dynamical systems
  12. A five-dimensional bilinear system
  13. Unicycle robot
  14. Husky robots
  15. Conclusion
  16. ...and 1 more sections

Key Result

Lemma 4.1

Given Assumption asmp2 and asmp1, suppose $p^{\text{ref}} \in C([0,T];L^2(\Omega))$, $\partial_t p^{\text{ref}} \in C([0,T];L^2(\Omega))$, $p^{\text{ref}}_t$ is a probability density and $p_t > 0$, for all $t\geq 0$. Then there exists a control law $\pi^i$ such that a solution $p^{\textup{c}}$ of th

Figures (5)

  • Figure 1: Unicycle robots navigating a non-convex environment to reach the standard Gaussian.
  • Figure 2: Reformulation from the classical control problem to the density control problem.
  • Figure 3: Experiments with a five-dimensional nonlinear system \ref{['eqn:fived-sys']}. (a) Final KL divergence for different number of measurement instances $N$ vs. training iterations with $M=2000$: validates Theorem \ref{['thm:tackdens']} that neural network can ensure $p^{\textup{c}}=p_\text{target}$, (b) Final KL divergence vs. training iterations with $M=300$: shows that with fewer measurement instances, we need more training samples to approximate densities, (c) Cost \ref{['eqn:approx-prob']} vs training iterations: depicts the sample complexity of the DDPM feedback control method.
  • Figure 4: Experiments with unicycle dynamics \ref{['eqn:unicycle']} with $p_\text{target} = \mathcal{N}(4,0.2I)$. (a) Final estimated KL divergence for different number of measurement instances $N$ vs. training iterations: shows that more measurement instances are required to achieve better feedback control when going from one Gaussian distribution to another, (b) Final KL divergence vs training iterations for different number of training samples: shows that the neural network can identify the controller with sufficiently large number of training samples and state measuring instances.
  • Figure 5: Experiments with Husky robots (Top) Husky robots with initial positions sampled from a uniform distribution, (Bottom) Final positions of the different robots sampled from a neighbourhood around the origin.

Theorems & Definitions (11)

  • Remark 1
  • Remark 2
  • Definition 1
  • Lemma 4.1: Exact tracking of positive densities
  • Theorem 4.2: Feedback control by tracking reverse process
  • Proposition A.1: Properties of the heat equation
  • proof
  • Proposition A.2
  • proof
  • Lemma A.3: Tracking the Heat Equation
  • ...and 1 more