Sim-to-Real Transfer via a Style-Identified Cycle Consistent Generative Adversarial Network: Zero-Shot Deployment on Robotic Manipulators through Visual Domain Adaptation

TL;DR

The paper tackles the high sample cost of DRL in industrial robotics by enabling zero-shot sim-to-real transfer through image-domain adaptation. It introduces StyleID-CycleGAN (SICGAN), which adds demodulated convolutions and an identity loss to CycleGAN to translate virtual observations into realistic real-synthetic images. A DRL agent trained in a virtual environment using SICGAN-translated inputs achieves robust zero-shot deployment on two different manipulators, with accuracies above 95% in most workspace regions. The approach is hardware-conscious, using a single externally mounted camera and AR-based evaluation, and demonstrates generalization to real objects with varying colors and shapes. This pipeline offers a scalable path to deploy DRL policies in industrial settings without additional real-world training.

Abstract

The sample efficiency challenge in Deep Reinforcement Learning (DRL) compromises its industrial adoption due to the high cost and time demands of real-world training. Virtual environments offer a cost-effective alternative for training DRL agents, but the transfer of learned policies to real setups is hindered by the sim-to-real gap. Achieving zero-shot transfer, where agents perform directly in real environments without additional tuning, is particularly desirable for its efficiency and practical value. This work proposes a novel domain adaptation approach relying on a Style-Identified Cycle Consistent Generative Adversarial Network (StyleID-CycleGAN or SICGAN), an original Cycle Consistent Generative Adversarial Network (CycleGAN) based model. SICGAN translates raw virtual observations into real-synthetic images, creating a hybrid domain for training DRL agents that combines virtual dynamics with real-like visual inputs. Following virtual training, the agent can be directly deployed, bypassing the need for real-world training. The pipeline is validated with two distinct industrial robots in the approaching phase of a pick-and-place operation. In virtual environments agents achieve success rates of 90 to 100\%, and real-world deployment confirms robust zero-shot transfer (i.e., without additional training in the physical environment) with accuracies above 95\% for most workspace regions. We use augmented reality targets to improve the evaluation process efficiency, and experimentally demonstrate that the agent successfully generalizes to real objects of varying colors and shapes, including LEGO\textsuperscript{\textregistered}~cubes and a mug. These results establish the proposed pipeline as an efficient, scalable solution to the sim-to-real problem.

Figures (29)

• Figure 1: General proposed pipeline for sim-to-real zero-shot transfer in industrial applications. The pipeline begins with stage 1, where the virtual environment is created, and the MDP is defined. Stage 2 outlines the steps required to train the SICGAN. Using this SICGAN, the DRL agent is trained using real-synthetic observations generated from raw virtual data. Once the agent is trained and evaluated in the virtual environment, it is deployed directly to the real environment via zero-shot transfer.
• Figure 2: IRB120 (Source: ABB ABB_IRB120_datasheet) (a) and UR3e (b) axes and their rotation.
• Figure 3: IRB120 workspace and axes location. The target coordinates must be within the intervals [0.2, 0.4] m for the $x$-axis and [--0.3, 0.3] m for the $y$-axis.
• Figure 4: Camera poses in the IRB120 setup: (a) around the $z$-axis, (b) around the $y$-axis.
• Figure 5: IRB120 virtual observations with 640$\times$640 pixel resolution (a) and 64$\times$64 pixel resolution (b), which is the one received by the agent.