Robust Multi-Modal Policies for Industrial Assembly via Reinforcement Learning and Demonstrations: A Large-Scale Study

TL;DR

The paper addresses the barrier to deploying reinforcement learning in industrial insertion tasks by proposing SHIELD, an industry-oriented RL-from-demonstrations framework built on DDPGfD. SHIELD integrates on-policy corrections, relative-coordinate formulations, pre-trained visual features, and multi-modal sensing to achieve high reliability and competitive cycle times on the NIST assembly benchmark, including moving-target and blind-insertion tasks. The large-scale evaluation demonstrates 99.8% success across 13K trials and close-to-human performance on dynamic HDMI insertion, with an open dataset to foster further research. This work suggests a practical pathway for bringing learning-based robotic insertion into real-world manufacturing with strong generalization and robustness."

Abstract

Over the past several years there has been a considerable research investment into learning-based approaches to industrial assembly, but despite significant progress these techniques have yet to be adopted by industry. We argue that it is the prohibitively large design space for Deep Reinforcement Learning (DRL), rather than algorithmic limitations per se, that are truly responsible for this lack of adoption. Pushing these techniques into the industrial mainstream requires an industry-oriented paradigm which differs significantly from the academic mindset. In this paper we define criteria for industry-oriented DRL, and perform a thorough comparison according to these criteria of one family of learning approaches, DRL from demonstration, against a professional industrial integrator on the recently established NIST assembly benchmark. We explain the design choices, representing several years of investigation, which enabled our DRL system to consistently outperform the integrator baseline in terms of both speed and reliability. Finally, we conclude with a competition between our DRL system and a human on a challenge task of insertion into a randomly moving target. This study suggests that DRL is capable of outperforming not only established engineered approaches, but the human motor system as well, and that there remains significant room for improvement. Videos can be found on our project website: https://sites.google.com/view/shield-nist.

Figures (9)

• Figure 1: An overview of our method: (1) gather human demonstration data by teleportation, (2) robot executes its current policy from the latest snapshot, (3) human perform occasional on-policy correction if necessary, (4) asynchronous training of an improved DDPGfD agent:SHIELD.
• Figure 2: NIST robotic challenge board setup: (a) Robot inserting a connector, (b) Three representative connectors.
• Figure 3: An illustration of on-policy correction: (a)the robot fails behind the sidewall of the male connector, (b)human engages by using remote teaching device, (c)human corrects robot's behavior
• Figure 4: Different modality inputs to the policy.
• Figure 5: An illustration of dynamic the insertion task.