Physical Simulation for Multi-agent Multi-machine Tending

TL;DR

A simplistic robotic system is leveraged to work with RL with real-world data without having to deploy large expensive robots in a manufacturing setting to provide an initial understanding of the real deployment challenges.

Abstract

The manufacturing sector was recently affected by workforce shortages, a problem that automation and robotics can heavily minimize. Simultaneously, reinforcement learning (RL) offers a promising solution where robots can learn through interaction with the environment. In this work, we leveraged a simplistic robotic system to work with RL with "real" data without having to deploy large expensive robots in a manufacturing setting. A real-world tabletop arena was designed with robots that mimic the agents' behavior in the simulation. Despite the difference in dynamics and machine size, the robots were able to depict the same behavior as in the simulation. In addition, those experiments provided an initial understanding of the real deployment challenges.

Figures (2)

• Figure 1: The real-world experiment arena: 3 Zooids, storage area (blue in the middle), machines (blue boxes), and machines' blockers (black)
• Figure 2: From the left: the episode total return, total delivered parts, and total collisions for AB-MAPPO (blue) compared to MAPPO (red)