Multi-Agent Systems for Robotic Autonomy with LLMs
Junhong Chen, Ziqi Yang, Haoyuan G Xu, Dandan Zhang, George Mylonas
TL;DR
The paper presents an LLM-powered multi-agent framework for robotic autonomy, decomposing task analysis, robot design, and reinforcement learning into three core agents that produce executable RL code and comprehensive reports. It demonstrates generalization across industrial and medical task scenarios and conducts ablation studies to quantify each agent's contribution, using GPT-4o-mini, DeepSeek-V3, GPT-4o, and DeepSeek-R1. Findings indicate the framework can generate feasible robot configurations and control policies when task prompts are sufficiently detailed, with DeepSeek-R1 showing the strongest overall performance due to its reasoning capabilities. The work highlights practical implications for accelerating robotic system development and points to future enhancements in dynamic environments, hierarchical agent architectures, and integrated vision-language capabilities.
Abstract
Since the advent of Large Language Models (LLMs), various research based on such models have maintained significant academic attention and impact, especially in AI and robotics. In this paper, we propose a multi-agent framework with LLMs to construct an integrated system for robotic task analysis, mechanical design, and path generation. The framework includes three core agents: Task Analyst, Robot Designer, and Reinforcement Learning Designer. Outputs are formatted as multimodal results, such as code files or technical reports, for stronger understandability and usability. To evaluate generalizability comparatively, we conducted experiments with models from both GPT and DeepSeek. Results demonstrate that the proposed system can design feasible robots with control strategies when appropriate task inputs are provided, exhibiting substantial potential for enhancing the efficiency and accessibility of robotic system development in research and industrial applications.