Master Rules from Chaos: Learning to Reason, Plan, and Interact from Chaos for Tangram Assembly

TL;DR

MRChaos (Master Rules from Chaos), a robust and general solution for learning assembly policies that can generalize to novel objects, indicates that radical generalization in robotic assembly can be achieved by learning in much simpler domains.

Abstract

Tangram assembly, the art of human intelligence and manipulation dexterity, is a new challenge for robotics and reveals the limitations of state-of-the-arts. Here, we describe our initial exploration and highlight key problems in reasoning, planning, and manipulation for robotic tangram assembly. We present MRChaos (Master Rules from Chaos), a robust and general solution for learning assembly policies that can generalize to novel objects. In contrast to conventional methods based on prior geometric and kinematic models, MRChaos learns to assemble randomly generated objects through self-exploration in simulation without prior experience in assembling target objects. The reward signal is obtained from the visual observation change without manually designed models or annotations. MRChaos retains its robustness in assembling various novel tangram objects that have never been encountered during training, with only silhouette prompts. We show the potential of MRChaos in wider applications such as cutlery combinations. The presented work indicates that radical generalization in robotic assembly can be achieved by learning in much simpler domains.

Figures (6)

• Figure 1: The task of tangram assembly from the silhouette is to use seven pieces to assemble an object according to a silhouette prompt; The image on the bottom right shows the cat assembled by MRChaos.
• Figure 2: System Overview. A: During training, a random target object, along with a silhouette image $I_s$, is generated. At time step $t$, the $I_{tc}$ is captured from a top-down camera and concatenated with $I_s$ as $o_t$. The agent receives the $o_t$ and outputs the action $a_t$ for the robot to place the $n_t$ piece. The agent finally receives rewards $r_t$ according to the visual difference and updates the policy with PPO; B: Deployment of MRChaos in the real world.
• Figure 3: (a) Silhouette examples of randomly generated objects. The top two rows are generated by random placing; the last row is generated with the extra step that adds gravitational force to attract pieces; (b) The relative coverage rate curve of our policy during training. The second stage curriculum begins at the orange line.
• Figure 4: Example of silhouette prompts and corresponding target objects in different task families.
• Figure 5: Qualitative results of tangram assembly.A: Sequences show MRChaos assembling objects with silhouette prompts. From top to bottom, the objects belong to the H-Normal, H-Hard, and H-Fiendish task families, respectively. B: A sequence shows the robot assembling objects through the behavior cloning method; C: Sequences demonstrate MRChaos is capable of adapting to the dynamic environment (i.e., the human push the square in the second trial).