DiPPeST: Diffusion-based Path Planner for Synthesizing Trajectories Applied on Quadruped Robots

TL;DR

DiPPeST extends a diffusion-based global planner with a zero-shot, image-driven local planner and a visual servoing execution stack to enable real-time, obstacle-avoiding navigation for quadruped robots. By tracking global-path features with Lucas-Kanade optical flow, selecting intermediate waypoints via median-distance and directional similarity, and using ROI-based pixel statistics to steer local goal selection, the approach refines trajectories frame-by-frame without additional training. Key results show $92\%$ success in standard environments and $88\%$ generalization in more complex scenes, plus $80\%$ real-world success on a Unitree Go1 with RealSense sensing, outperforming two diffusion- or geometry-based baselines in narrow passages. The work demonstrates practical image-conditioned diffusion for mobile robotics, highlighting strengths in adaptability and zero-shot transfer while identifying avenues to improve re-planning speed and kinodynamics for broader applicability.

Abstract

We present DiPPeST, a novel image and goal conditioned diffusion-based trajectory generator for quadrupedal robot path planning. DiPPeST is a zero-shot adaptation of our previously introduced diffusion-based 2D global trajectory generator (DiPPeR). The introduced system incorporates a novel strategy for local real-time path refinements, that is reactive to camera input, without requiring any further training, image processing, or environment interpretation techniques. DiPPeST achieves 92% success rate in obstacle avoidance for nominal environments and an average of 88% success rate when tested in environments that are up to 3.5 times more complex in pixel variation than DiPPeR. A visual-servoing framework is developed to allow for real-world execution, tested on the quadruped robot, achieving 80% success rate in different environments and showcasing improved behavior than complex state-of-the-art local planners, in narrow environments.

Figures (6)

• Figure 1: DiPPeST: (top) Consecutive robot camera frames with the overlaid generated global (red) and local (green) trajectories, captured in real-time; (bottom) A quadruped robot following the planned path. Blurriness is due to robot fast motion.
• Figure 2: Examples of DiPPeR's training dataset: $100\times100$ random solvable maps with examples of end-to-end trajectories, generated through $A^{*}$.
• Figure 3: DiPPeST process of generating the local path and correcting global path failure. DiPPeST takes an input camera frame (A) and utilizes DiPPeR to generate a global path (B) and the waypoints. These are tracked at each input frame (C), and the optimal waypoint is selected. At each frame a, ROI is created around the current position and the waypoint and the optimal local goal position (D) within the ROI is selected (red cross) to avoid assigning the goal close to the obstacle due to (C) waypoint suboptimal positioning. DiPPeR is then used to generate a path to the goal position (E). Low resolution images are displayed representing the exact output of the diffusion model.
• Figure 4: Visual servoing framework for real-world robot path execution. DiPPeST 2D trajectories within RGB frame are de-projected into 3D world coordinates. The path follower translates the trajectories into velocity commands, and the robot module into motor commands to drive the robot towards the desired path.
• Figure 5: The effect of variation of a) floor and obstacle color, b) input image size, and c) camera PoV, over DiPPeST $\%$ success rate.