Subassembly to Full Assembly: Effective Assembly Sequence Planning through Graph-based Reinforcement Learning

TL;DR

The paper tackles the combinatorial complexity of robotic assembly sequence planning by formulating ASP as a graph-based Markov decision process and introducing Subassembly to Assembly (S2A), a graph neural network–driven reinforcement learning framework. It uses a volumetric voxel encoding of subassemblies and a GATv2-based encoder to predict Q-values over admissible edges, with a delayed reward strategy that rewards only actions leading toward a complete assembly, guided by a Double Deep Q-Network. The approach integrates physics-based feasibility checks via $\mathrm{RRT}^*$ and PyBullet during training and inference, enabling reliable planning under kinematic and gravity constraints. Empirical results show that S2A outperforms baselines across problem sizes, generalizes to unseen instances, and demonstrates a real-world robotic demonstration, indicating potential for scalable, automated ASP in complex assemblies.

Abstract

This paper proposes an assembly sequence planning framework, named Subassembly to Assembly (S2A). The framework is designed to enable a robotic manipulator to assemble multiple parts in a prespecified structure by leveraging object manipulation actions. The primary technical challenge lies in the exponentially increasing complexity of identifying a feasible assembly sequence as the number of parts grows. To address this, we introduce a graph-based reinforcement learning approach, where a graph attention network is trained using a delayed reward assignment strategy. In this strategy, rewards are assigned only when an assembly action contributes to the successful completion of the assembly task. We validate the framework's performance through physics-based simulations, comparing it against various baselines to emphasize the significance of the proposed reward assignment approach. Additionally, we demonstrate the feasibility of deploying our framework in a real-world robotic assembly scenario.

Figures (8)

• Figure 1: Demonstration of Jenga assembly using a robotic manipulator.
• Figure 2: (a) An example of assembling $3$ Tetris-like blocks, and (b) an illustration of an ASP problem represented as a directed graph.
• Figure 3: Pybullet simulation showcasing the assembly of a 7-part block.
• Figure 4: An overview of the S2A framework: Our framework takes $N$ volumetric representations of assembly states as input, applies $L$ rounds of message-passing procedures, and predicts Q-values to estimate the quality of admissible assembly actions. Starting from the root, a solution to the ASP problem is constructed by iteratively selecting the action with the highest Q-value.
• Figure 5: (a) A graph illustrating the success rate improvement of our method as the beam width increases, and (b) Comparison of training curves between two GNN-based approaches for the $7$-part assembly problem.