AssemblyComplete: 3D Combinatorial Construction with Deep Reinforcement Learning

TL;DR

This paper proposes a two-part deep reinforcement learning (DRL) framework that tackles teaching the robot to understand the objective of an incomplete assembly and learning a construction policy to complete the assembly, and demonstrates the framework's feasibility and robustness in a variety of assembly scenarios.

Abstract

A critical goal in robotics and autonomy is to teach robots to adapt to real-world collaborative tasks, particularly in automatic assembly. The ability of a robot to understand the original intent of an incomplete assembly and complete missing features without human instruction is valuable but challenging. This paper introduces 3D combinatorial assembly completion, which is demonstrated using combinatorial unit primitives (i.e., Lego bricks). Combinatorial assembly is challenging due to the possible assembly combinations and complex physical constraints (e.g., no brick collisions, structure stability, inventory constraints, etc.). To address these challenges, we propose a two-part deep reinforcement learning (DRL) framework that tackles teaching the robot to understand the objective of an incomplete assembly and learning a construction policy to complete the assembly. The robot queries a stable object library to facilitate assembly inference and guide learning. In addition to the robot policy, an action mask is developed to rule out invalid actions that violate physical constraints for object-oriented construction. We demonstrate the proposed framework's feasibility and robustness in a variety of assembly scenarios in which the robot satisfies real-life assembly with respect to both solution and runtime quality. Furthermore, results demonstrate that the proposed framework effectively infers and assembles incomplete structures for unseen and unique object types.

Figures (5)

• Figure 1: Our method is designed to: (a) Represent an incomplete structure as a voxel and point cloud. (b) Incorporate a library of real-world objects for inference and matching. In this step, target scaling with the best-matched complete object from the library and 3D point cloud registration are performed. 1 represents the incomplete/source point cloud. 2 represents the completed/target point cloud. (c) Train a DRL agent to use heuristics and satisfy multiple constraints for efficient sequential decision-making. Here, given the current state $s_t$ (incomplete structure being built, reference structure, and inventory), the PPO agent learns to choose an optimal action and improve its policy and value function using a reward.
• Figure 2: Real-World Physical Constraints. Red: invalid placement. Green: valid placement.
• Figure 3: Top row: Incomplete structure input. Middle: Stable matched reference object; not to original scale/rotation. Bottom: Visualized Result. Only \ref{['fig:h1']} has been seen before in training; the agent successfully completes other unseen incomplete structures.
• Figure 4: Example of sequential assembly with analysis. Top: Stability Analysis, White-Collapsing, Red-Risky, Black-Stable. Bottom: Real-world Lego Assembly, each unique color is a unique brick type.
• Figure 5: Comparisons of incomplete 3D Lego assemblies with various degrees of missing features with the same reference assembly. Top: Incomplete assemblies with missing features. Bottom: Finished assembly.