Dynamics-Guided Diffusion Model for Sensor-less Robot Manipulator Design

TL;DR

DGDM introduces dynamics-guided diffusion to automate task-specific, sensor-less manipulator design without task-specific training. It represents tasks as interaction profiles and learns a dynamics network to predict object motion under finger interaction, then uses a diffusion model guided by gradients of a task objective to generate tailored finger geometries. Across simple to complex 2D/3D tasks, DGDM outperforms unguided diffusion and gradient-based baselines, with faster design times (~0.8s) and robust sim-to-real transfer. This enables rapid, data-driven exploration and adoption of customized mechanical designs for robotic manipulation.

Abstract

We present Dynamics-Guided Diffusion Model (DGDM), a data-driven framework for generating task-specific manipulator designs without task-specific training. Given object shapes and task specifications, DGDM generates sensor-less manipulator designs that can blindly manipulate objects towards desired motions and poses using an open-loop parallel motion. This framework 1) flexibly represents manipulation tasks as interaction profiles, 2) represents the design space using a geometric diffusion model, and 3) efficiently searches this design space using the gradients provided by a dynamics network trained without any task information. We evaluate DGDM on various manipulation tasks ranging from shifting/rotating objects to converging objects to a specific pose. Our generated designs outperform optimization-based and unguided diffusion baselines relatively by 31.5% and 45.3% on average success rate. With the ability to generate a new design within 0.8s, DGDM facilitates rapid design iteration and enhances the adoption of data-driven approaches for robot mechanism design. Qualitative results are best viewed on our project website https://dgdm-robot.github.io/.

Figures (13)

• Figure 1: Task-specific Designs without Task-specific Training. Given different input objects (1st column), DGDM generates diverse manipulator geometries tailored to different manipulation tasks without task-specific training, which can be deployed under the sensor-less setting with an open-loop parallel closing motion.
• Figure 2: The Convergence Task is to design fingers that always reorient a target object to a specified orientation $\theta_{\textrm{target}}$ (in the manipulator frame) when closing the gripper in parallel. This task enables funneling objects from arbitrary poses to a specific $\theta_{\textrm{target}}$ in a sensor-less setting, and moving objects to any particular configuration combined with a global transformation of the gripper. Despite its utility, designing for convergence can be counter-intuitive -- it often takes an expert many design cycles to come up with just one design for one object. In contrast, DGDM can generate a functional design for a new object in seconds.
• Figure 3: DGDM generates finger shapes given a target object and task, specified as a target interaction profile (§ \ref{['sec:task']}). This is compared with the dynamics network's prediction of the current interaction profile, which is used to construct an objective (§ \ref{['sec:dynamics']}). Gradients of the objective iteratively guide the reverse denoising process of a manipulator shape diffusion model (§ \ref{['sec:diffusion']}).
• Figure 4: Effect of Scaled Guidance. From left to right we increase the scaled guidance in the diffusion process. The increased scale enforces more task guidance and achieves higher task performance (shifting down) while reducing the diversity of generated designs.
• Figure 5: Convergence Results. For each pair of finger designs, we show the range of initial orientations ("cvrg. range") which converges to the same convergence mode ("cvrg. target").