Accelerated Multi-Modal Motion Planning Using Context-Conditioned Diffusion Models

TL;DR

This work tackles the generalization gap in diffusion-based motion planning by introducing Context-Aware Motion Planning Diffusion (CAMPD), a diffusion model conditioned on sensor-agnostic contextual information via time and context encoders with an attention-enabled U-Net. Framed as planning-by-inference, CAMPD uses classifier-free guidance to produce collision-free, multi-modal trajectories for high-DoF robots, demonstrated on a 7-DoF Panda with real-time capability. The key contributions are: environment-agnostic context conditioning that handles arbitrary obstacle counts, improved generalization to unseen environments, and fast generation of executable trajectories without post-processing. The results show CAMPD outperforms state-of-the-art baselines on unseen scenes and real-world-inspired tasks, highlighting its potential for online planning and replanning in dynamic, complex settings.

Abstract

Classical methods in robot motion planning, such as sampling-based and optimization-based methods, often struggle with scalability towards higher-dimensional state spaces and complex environments. Diffusion models, known for their capability to learn complex, high-dimensional and multi-modal data distributions, provide a promising alternative when applied to motion planning problems and have already shown interesting results. However, most of the current approaches train their model for a single environment, limiting their generalization to environments not seen during training. The techniques that do train a model for multiple environments rely on a specific camera to provide the model with the necessary environmental information and therefore always require that sensor. To effectively adapt to diverse scenarios without the need for retraining, this research proposes Context-Aware Motion Planning Diffusion (CAMPD). CAMPD leverages a classifier-free denoising probabilistic diffusion model, conditioned on sensor-agnostic contextual information. An attention mechanism, integrated in the well-known U-Net architecture, conditions the model on an arbitrary number of contextual parameters. CAMPD is evaluated on a 7-DoF robot manipulator and benchmarked against state-of-the-art approaches on real-world tasks, showing its ability to generalize to unseen environments and generate high-quality, multi-modal trajectories, at a fraction of the time required by existing methods.

Figures (5)

• Figure 1: Overview of CAMPD: During training, trajectories are perturbed with Gaussian noise at a random diffusion step, while start and goal states remain fixed for conditioning. Context is encoded and randomly dropped (Bernoulli $p_d$) for conditional/unconditional training. A U-Net with attention predicts the noise using MSE loss. During inference, noise is iteratively denoised via reverse diffusion with classifier-free guidance ($w$), keeping start and goal fixed. Finally, a Gaussian filter is applied to the resulting trajectory to reduce jerk.
• Figure 2: Context Encoder. Each instance $\mathbf{c}_{k,l}$ of context type $k$ is processed by a dedicated multi-layer perceptron (MLP)
• Figure 3: Attention Mechanism. The encoded context vectors $\mathbf{z}{\mathbf{c}_{k,l}}$ and the latent time step representation $\mathbf{z}_t$ are fused into the U-Net through a residual architecture of self- and cross-attention layers.
• Figure 4: Example of multi-modal trajectories generated by CAMPD in an unseen environment with spherical obstacles (blue). The white and purple arms indicate start and goal configurations, respectively. The yellow line shows the end-effector trajectory, with (c) highlighting the best trajectory of a batch of 100 samples.
• Figure 5: Feasible trajectories generated by CAMPD for three previously unseen problems from the $\text{M}\pi\text{Net}$mpinets test set.