Constraint-Aware Diffusion Guidance for Robotics: Real-Time Obstacle Avoidance for Autonomous Racing

TL;DR

Safety-constrained real-time planning with diffusion models is challenging; CoDiG augments the reverse diffusion with a barrier-based potential so that $p_t(x_t|\mathcal{C}) = p_t(x_t) e^{-\gamma_t V(x_t;\mathcal{C})}/Z_t$ and its score is $\nabla_x \log p_t(x_t|\mathcal{C}) = \nabla_x \log p_t(x_t) - \gamma_t \nabla_x V(x_t;\mathcal{C})$, with a warm-start to speed up inference. The contributions include a general constraint-aware diffusion framework, a barrier-based inference mechanism, and a real-world autonomous racing demonstration with dynamic obstacles and 2.5 Hz planning, illustrating near time-optimal behavior under safety constraints. This work enables robust, real-time, constraint-satisfying diffusion-based planning across safety-critical robotic domains.

Abstract

Diffusion models hold great potential in robotics due to their ability to capture complex, high-dimensional data distributions. However, their lack of constraint-awareness limits their deployment in safety-critical applications. We propose Constraint-Aware Diffusion Guidance (CoDiG), a data-efficient and general-purpose framework that integrates barrier functions into the denoising process, guiding diffusion sampling toward constraint-satisfying outputs. CoDiG enables constraint satisfaction even with limited training data and generalizes across tasks. We evaluate our framework in the challenging setting of miniature autonomous racing, where real-time obstacle avoidance is essential. Real-world experiments show that CoDiG generates safe outputs efficiently under dynamic conditions, highlighting its potential for broader robotic applications. A demonstration video is available at https://youtu.be/KNYsTdtdxOU.

Figures (9)

• Figure 1: Intermediate denoising results during sampling at three representative time steps $t = \qty{1}{\second},~\qty{0.591}{\second},~\qty{0.002}{\second}$, from left to right. (a) Sampling process without the barrier function. (b) Sampling process with the proposed barrier function. Black dots and arrows represent the generated trajectory points and their heading directions in the global frame.
• Figure 2: Real-world demonstration of real-time obstacle avoidance using CoDiG. Red lines represent the planned trajectories generated by the CoDiG diffusion planner. Gray circles indicate static obstacles, and black circles represent dynamic obstacles. Black dashed lines show the predicted trajectory from the TMPC while following the reference plan. Each episode illustrates a complete avoidance cycle: obstacle encroachment, real-time replanning, and successful passage.
• Figure 3: (a) A time-optimal trajectory (red line) computed for a given obstacle configuration (gray regions) on the racing track. Black rectangles and arrows indicate the approximate vehicle shape and heading. (b) Redundant obstacles (brown regions) added in areas that do not affect the trajectory, providing data augmentation without solving additional optimal control problems. (c) A flattened track representation in the local Frenet coordinate system, visualizing both the trajectory (red line) and obstacles (black regions).
• Figure 4: Architecture of the proposed time-conditioned score-based generative model. The U-Net backbone extracts multi-scale features through a sequence of convolutional and deconvolutional layers, with temporal embeddings injected via dense layers. Spatial transformer modules enable conditional attention guided by task-specific context. Skip connections ensure spatial consistency across scales.
• Figure 5: Training performance of the diffusion model under different input configurations and noise schedules. From left to right, the three plots correspond to using (i) lateral displacement $\hat{y}$ in the Frenet frame only, (ii) lateral displacement $\hat{y}$ and yaw angle $\widehat{\phi}$ in the Frenet frame, and (iii) the states $\hat{x},~\hat{y},~\widehat{\phi}$ along with velocities $\hat{v}_x,~\hat{v}_y,~\widehat{\omega}$ as model inputs. Each setting was trained for 500 epochs while varying the parameters $r_1$ and $r_0$ in the noise schedule.