Compositional Diffusion-Based Continuous Constraint Solvers

TL;DR

The paper addresses solving complex continuous constraint satisfaction problems in robotic planning by introducing Diffusion-CCSP, which learns modular diffusion models for individual constraint types and composes them via energy minimization over a constraint graph. This framework enables solving CCSPs with heterogeneous constraints (geometric, physical, qualitative) and generalizes to novel constraint combinations and larger problem instances. Training uses a denoising energy-based objective, and inference employs annealed Langevin dynamics, enabling conditioning on subsets of variables. Empirical results across 2D and 3D packing, arrangement, and stacking tasks demonstrate strong generalization and efficiency, outperforming rejection-sampling baselines and StructDiffusion, and enabling integration with task-and-motion planning. Overall, Diffusion-CCSP provides a scalable, general-purpose approach for learning-based constraint solving in robotic manipulation with practical TAMP implications.

Abstract

This paper introduces an approach for learning to solve continuous constraint satisfaction problems (CCSP) in robotic reasoning and planning. Previous methods primarily rely on hand-engineering or learning generators for specific constraint types and then rejecting the value assignments when other constraints are violated. By contrast, our model, the compositional diffusion continuous constraint solver (Diffusion-CCSP) derives global solutions to CCSPs by representing them as factor graphs and combining the energies of diffusion models trained to sample for individual constraint types. Diffusion-CCSP exhibits strong generalization to novel combinations of known constraints, and it can be integrated into a task and motion planner to devise long-horizon plans that include actions with both discrete and continuous parameters. Project site: https://diffusion-ccsp.github.io/

Figures (20)

• Figure 1: Solving Continuous Constraint Satisfaction Problems by Composing Diffusion Models. Our approach combines diffusion models, representing individual constraints, to generate object placement poses. The choice of an object's placement depends on both qualitative constraints about object placement and collision avoidance constraints on the object and the robot gripper.
• Figure 2: Continuous Constraint Satisfaction Problem (CCSP) in Robot Planning. It unifies geometric, physical, and qualitative constraints. To place A into the tray, we need to generate the grasping pose $\textit{grasp}_A$, placement pose $\textit{pose}_A$, and the robot arm trajectory. We omit collision-free constraints with robots in (b) for brevity.
• Figure 3: Illustration of the denoising neural network for satisfying constraint $\textit{cfree}$ (collision-free)
• Figure 4: Illustration of four domains. They contain geometric, physical, and qualitative constraints. (a) Triangle packing. (b) Dense 2D packing with qualitative constraints. The figure shows a subset of 45 constraints of 13 types. (c) 3D object stacking. The arrows show the support relationships. (d) 3D object packing with a panda robot.
• Figure 5: Quantitative Comparisons of Constraint Solvers. Accumulated number of problems solved in 10 runs of different models. OOD=Out of training distribution. The shaded area indicates the standard deviation of various models across five different seeds. The sequential sampling baseline completely failed for task (c) and hard tasks in (b) and (d). Our full model performs better than a variant without ULA sampling and better than StructDiffusion in more complex problems (d).