Building Gradient by Gradient: Decentralised Energy Functions for Bimanual Robot Assembly

TL;DR

This work introduces Building Gradient by Gradient (BGBG), a decentralised gradient-based planning framework for bimanual assembly that uses piecewise continuous energy functions to automatically generate sub-goals and enable rapid replanning without long-horizon task planning. By per-arm optimization and a gradient composition mechanism, the method yields emergent coordination, autonomous handovers, and robustness to disturbances in tight-tolerance insertions. The authors formalize the problem as energy-function optimization, demonstrate convergence and fast planning in simulation, and validate real-world performance on a dual-arm Kinova platform with several 44-beam assemblies, achieving millimetre-scale alignment with limitations due to joint stiction. They also compare against MRPP and RAMP SAT, showing faster per-component planning and demonstrating real-time adaptability in human-robot collaboration.

Abstract

There are many challenges in bimanual assembly, including high-level sequencing, multi-robot coordination, and low-level, contact-rich operations such as component mating. Task and motion planning (TAMP) methods, while effective in this domain, may be prohibitively slow to converge when adapting to disturbances that require new task sequencing and optimisation. These events are common during tight-tolerance assembly, where difficult-to-model dynamics such as friction or deformation require rapid replanning and reattempts. Moreover, defining explicit task sequences for assembly can be cumbersome, limiting flexibility when task replanning is required. To simplify this planning, we introduce a decentralised gradient-based framework that uses a piecewise continuous energy function through the automatic composition of adaptive potential functions. This approach generates sub-goals using only myopic optimisation, rather than long-horizon planning. It demonstrates effectiveness at solving long-horizon tasks due to the structure and adaptivity of the energy function. We show that our approach scales to physical bimanual assembly tasks for constructing tight-tolerance assemblies. In these experiments, we discover that our gradient-based rapid replanning framework generates automatic retries, coordinated motions and autonomous handovers in an emergent fashion.

Figures (7)

• Figure 1: A decentralised gradient-based planning approach for real-world bimanual assembly. Each robot creates a piecewise continuous energy function through the automatic switching between myopic functions. The planner facilitates rapid replanning, where reattempts, coordination between arms and human/robot collaboration emerge naturally. A summary of the alogrithm and results is found at https://youtu.be/Kw99FtEhZB8.
• Figure 2: The left sub-figure contains a computational graph showing a set of energy functions for assembly. There is an energy function for describing the contact dynamics between any of the $I$ hands ${\bm{h}}_i$ and any of the $N$ components ${\bm{c}}_n$. The contact value function is composed with the goal loss. The goal loss is defined as the mean-square error between the component's current pose ${\bm{c}}_n$ and its final goal pose ${\bm{c}}^*_n$. The right sub-figure shows how gradients are composed together, creating a set of motions for each hand that, once enacted, will solve the assembly task.
• Figure 3: Goal loss for 50 experiments. The loss drops when a hand moves a component to its correct location. Once in the correct location, a new task is automatically selected, and the hands move to a new component. During the motion to a new component, the loss remains constant.
• Figure 4: Our approach optimises both subgoals and motions concurrently using gradient descent for assembly. Starting at the top row, the robot grasps a component in each hand. These are moved to their target locations. The right arm regrasps its component to make slight adjustments to its pose in the top right figure. Once the first two pieces are correctly located, the robot moves its arms to other components for assembly, as shown in the middle row. The bottom row shows the insertion of the last two components. Coordination arises between the arms automatically as one component disturbs another. As the left hand adjusts the final pose of both components, the left arm retracts, as shown in the bottom row, middle figure. A video of our results can be found at the following link: https://youtu.be/Kw99FtEhZB8.
• Figure 5: Timing comparison for planning assembly motions. BGBG is our method, MRPP represents hartmann2025multirobotplanning and SAT is the SAT solver from collins2023ramp. The MRPP Pick does not consider component mating, whilst MRPP TAMP version does. Lower planning durations lead to faster task replanning for reattempts during assembly.