We Choose to Go to Space: Agent-driven Human and Multi-Robot Collaboration in Microgravity
Miao Xin, Zhongrui You, Zihan Zhang, Taoran Jiang, Tingjia Xu, Haotian Liang, Guojing Ge, Yuchen Ji, Shentong Mo, Jian Cheng
TL;DR
SpaceAgents-1 addresses the challenge of coordinating humans and multiple robots in microgravity by coupling a foundation-model-guided planning arena with embodied Skill-Expert Agents, enabling long-horizon HMRC tasks within SpaceSim. The authors introduce SpaceSim, a microgravity physics platform built on SAPIEN, and a three-robot intra-cabin robot catalog (Free-flying, Rail-type, Dexterous) with real2sim HRI. The system operates through a Planner–Actor–Discriminator loop, where GPT-4 outputs a directed collaboration graph, SEAs execute skills via PPO-trained policies, and a Discriminator evaluates task state via a vision-language model, feeding back to the DMA. Results show that SpaceAgents-1 offers planning performance comparable to humans while facing skill-execution gaps under microgravity, underscoring the need for microgravity-specific skill learning and the value of an open-source HMRC platform for future research.
Abstract
We present SpaceAgents-1, a system for learning human and multi-robot collaboration (HMRC) strategies under microgravity conditions. Future space exploration requires humans to work together with robots. However, acquiring proficient robot skills and adept collaboration under microgravity conditions poses significant challenges within ground laboratories. To address this issue, we develop a microgravity simulation environment and present three typical configurations of intra-cabin robots. We propose a hierarchical heterogeneous multi-agent collaboration architecture: guided by foundation models, a Decision-Making Agent serves as a task planner for human-robot collaboration, while individual Skill-Expert Agents manage the embodied control of robots. This mechanism empowers the SpaceAgents-1 system to execute a range of intricate long-horizon HMRC tasks.