MissionGPT: Mission Planner for Mobile Robot based on Robotics Transformer Model

TL;DR

This approach demonstrates the possibility of setting a task for a mobile robot and its successful execution without the use of perception algorithms, based only on the data coming from the camera, and can be scaled for any type of robot and for any number of robots.

Abstract

This paper presents a novel approach to building mission planners based on neural networks with Transformer architecture and Large Language Models (LLMs). This approach demonstrates the possibility of setting a task for a mobile robot and its successful execution without the use of perception algorithms, based only on the data coming from the camera. In this work, a success rate of more than 50\% was obtained for one of the basic actions for mobile robots. The proposed approach is of practical importance in the field of warehouse logistics robots, as in the future it may allow to eliminate the use of markings, LiDARs, beacons and other tools for robot orientation in space. In conclusion, this approach can be scaled for any type of robot and for any number of robots.

Figures (11)

• Figure 1: Experimental setup view
• Figure 2: Hardware system architecture
• Figure 3: Software system architecture
• Figure 4: Data sample example
• Figure 5: Encoder-only model scheme