Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Diffusion Predictive Control with Constraints

Ralf Römer, Alexander von Rohr, Angela P. Schoellig

TL;DR

This work addresses the challenge of enforcing hard, novel constraints on diffusion-based policies in robotics. It introduces Diffusion Predictive Control with Constraints (DPCC), which injects model-based projections into the diffusion denoising process and applies constraint tightening to account for model mismatch, enabling predictive control with constraint satisfaction. The approach yields dynamically feasible, goal-reaching trajectories under unseen constraints and is demonstrated to outperform baselines in a robot manipulation task with obstacles. The main contributions are the projection-based conditioning of diffusion trajectories, a constraint-tightening mechanism for robustness, and trajectory-selection schemes that preserve task performance while ensuring safety.

Abstract

Diffusion models have become popular for policy learning in robotics due to their ability to capture high-dimensional and multimodal distributions. However, diffusion policies are stochastic and typically trained offline, limiting their ability to handle unseen and dynamic conditions where novel constraints not represented in the training data must be satisfied. To overcome this limitation, we propose diffusion predictive control with constraints (DPCC), an algorithm for diffusion-based control with explicit state and action constraints that can deviate from those in the training data. DPCC incorporates model-based projections into the denoising process of a trained trajectory diffusion model and uses constraint tightening to account for model mismatch. This allows us to generate constraint-satisfying, dynamically feasible, and goal-reaching trajectories for predictive control. We show through simulations of a robot manipulator that DPCC outperforms existing methods in satisfying novel test-time constraints while maintaining performance on the learned control task.

Diffusion Predictive Control with Constraints

TL;DR

This work addresses the challenge of enforcing hard, novel constraints on diffusion-based policies in robotics. It introduces Diffusion Predictive Control with Constraints (DPCC), which injects model-based projections into the diffusion denoising process and applies constraint tightening to account for model mismatch, enabling predictive control with constraint satisfaction. The approach yields dynamically feasible, goal-reaching trajectories under unseen constraints and is demonstrated to outperform baselines in a robot manipulation task with obstacles. The main contributions are the projection-based conditioning of diffusion trajectories, a constraint-tightening mechanism for robustness, and trajectory-selection schemes that preserve task performance while ensuring safety.

Abstract

Diffusion models have become popular for policy learning in robotics due to their ability to capture high-dimensional and multimodal distributions. However, diffusion policies are stochastic and typically trained offline, limiting their ability to handle unseen and dynamic conditions where novel constraints not represented in the training data must be satisfied. To overcome this limitation, we propose diffusion predictive control with constraints (DPCC), an algorithm for diffusion-based control with explicit state and action constraints that can deviate from those in the training data. DPCC incorporates model-based projections into the denoising process of a trained trajectory diffusion model and uses constraint tightening to account for model mismatch. This allows us to generate constraint-satisfying, dynamically feasible, and goal-reaching trajectories for predictive control. We show through simulations of a robot manipulator that DPCC outperforms existing methods in satisfying novel test-time constraints while maintaining performance on the learned control task.

Paper Structure

This paper contains 15 sections, 2 theorems, 20 equations, 3 figures, 3 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem Statement
  4. Background on Trajectory Diffusion
  5. Methodology
  6. Diffusion-Based Receding Horizon Control
  7. Constraint- and Model-Based Trajectory Diffusion
  8. Constraint Tightening
  9. Trajectory Selection
  10. Evaluation and Discussion
  11. Setup
  12. Results
  13. Conclusion
  14. Training Details
  15. Testing Details

Key Result

theorem 1

Let $\mathcal{Z}_{\bm{f}}$ be a closed convex set, $\sigma_k > 0$, $\forall k \in \mathbb{I}_{1}^K$, and let the feasibility likelihood be defined by ${p(\mathcal{O} = 1 | \bm{\tau}, k) \propto \exp{\left(-\frac{1}{2\sigma_k^2}d(\bm{\tau}, \mathcal{Z}_{\bm{f}})^2\right)}}$, where ${d(\bm{\tau}, \mat where the learned mean ${\bm{\mu}_{\bm{\theta}}^k = \bm{\mu}_{\bm{\theta}}(\bm{\tau}^k,k)}$ is the

Figures (3)

  • Figure 1: Experiments: (a) Simulation environment, where the objective is to reach the green line with the end-effector without collisions. (b) Multimodal trajectory distribution in the training dataset. (c) Novel test-time constraints (blue).
  • Figure 2: Impact of our constraint tightening method and the trajectory selection criterion (DPCC-R: random, DPCC-T: temporal consistency, DPCC-C: cumulative projection cost) on the success rates and the number of timesteps needed.
  • Figure 3: Closed-loop trajectories with DPCC for different trajectory selection criteria and five training seeds, which are indicated by the trajectories' colors. The tightened constraints are visualized in light blue. With our proposed trajectory selection criteria (DPCC-T and DPCC-C), we obtain a smoother behavior and shorter time to reach the goal.

Theorems & Definitions (4)

  • theorem 1
  • proof
  • theorem 2
  • proof