ARO: Large Language Model Supervised Robotics Text2Skill Autonomous Learning

TL;DR

The paper addresses the high cost and limited scalability of human-led robotics learning by introducing ARO, an LLM-guided framework that autonomously designs reward functions, evaluates performance, and refines policies to acquire robot skills. Through a five-module loop (RFG, Training, EFG, PE, EE) and an SAC-based training regime, ARO translates natural language instructions into executable RL agents and iteratively improves them based on GPT-4-derived evaluations. Experiments in MuJoCo-based gym environments with Humanoid, Hopper, and HalfCheetah demonstrate autonomous skill learning across 29 prompts and 18 successful trials, illustrating the potential to reduce human intervention in robot skill development. The work also discusses limitations in environmental understanding and prompt sensitivity, offering a roadmap for improving robustness and generalization in autonomous robot learning.

Abstract

Robotics learning highly relies on human expertise and efforts, such as demonstrations, design of reward functions in reinforcement learning, performance evaluation using human feedback, etc. However, reliance on human assistance can lead to expensive learning costs and make skill learning difficult to scale. In this work, we introduce the Large Language Model Supervised Robotics Text2Skill Autonomous Learning (ARO) framework, which aims to replace human participation in the robot skill learning process with large-scale language models that incorporate reward function design and performance evaluation. We provide evidence that our approach enables fully autonomous robot skill learning, capable of completing partial tasks without human intervention. Furthermore, we also analyze the limitations of this approach in task understanding and optimization stability.

Figures (2)

• Figure 1: ARO's Framework
• Figure 2: Repeatedly perform squatting and standing actions by Humanoid