Physics-Aware Combinatorial Assembly Sequence Planning using Data-free Action Masking

TL;DR

This work tackles assembly sequence planning for combinatorial block assembly under physical feasibility constraints. It formulates ASP as an MDP and uses deep reinforcement learning with an online, data-free physics-aware action mask to prune invalid actions, ensuring feasible plans without relying on a physics simulator. Across hand-crafted shapes, self-generated shapes, StableLego designs, and robot-execution demonstrations, the approach achieves complete success where baselines falter, including scenarios requiring intricate stability and operability considerations. The action mask encodes task, collision, inventory, operability, and stability constraints, enabling practical, robot-ready planning and extending to other block-assembly domains with minimal supervision.

Abstract

Combinatorial assembly uses standardized unit primitives to build objects that satisfy user specifications. This paper studies assembly sequence planning (ASP) for physical combinatorial assembly. Given the shape of the desired object, the goal is to find a sequence of actions for placing unit primitives to build the target object. In particular, we aim to ensure the planned assembly sequence is physically executable. However, ASP for combinatorial assembly is particularly challenging due to its combinatorial nature. To address the challenge, we employ deep reinforcement learning to learn a construction policy for placing unit primitives sequentially to build the desired object. Specifically, we design an online physics-aware action mask that filters out invalid actions, which effectively guides policy learning and ensures violation-free deployment. In the end, we apply the proposed method to Lego assembly with more than 250 3D structures. The experiment results demonstrate that the proposed method plans physically valid assembly sequences to build all structures, achieving a $100\%$ success rate, whereas the best comparable baseline fails more than $40$ structures. Our implementation is available at \url{https://github.com/intelligent-control-lab/PhysicsAwareCombinatorialASP}.

Figures (10)

• Figure 1: Illustration of physics-aware combinatorial assembly.
• Figure 2: Overview of the proposed physics-aware ASP pipeline. In policy training, the physics-aware action mask filters out invalid actions and guides policy learning. The agent interacts with the environment by sampling actions from the masked policy distribution and learns from the reward feedback. In policy deployment, the action is selected by maximum likelihood on the masked policy distribution to ensure violation-free deployment.
• Figure 3: Training rewards for different methods on different evaluation sets.
• Figure 4: Example hand-crafted shapes and their planned assemblies in real.
• Figure 5: Examples of self-supervised shapes and their real assemblies.