Probabilistically Correct Language-based Multi-Robot Planning using Conformal Prediction
Jun Wang, Guocheng He, Yiannis Kantaros
TL;DR
This work tackles language-based mission planning for robot teams by introducing S-ATLAS, a distributed planner that uses conformal prediction to create local, uncertainty-aware prediction sets for each robot. By solving sequential MCQA problems and selectively seeking help when predictions are uncertain, S-ATLAS achieves user-defined mission success rates with reduced reliance on human input and improved scalability over centralized CP baselines. The authors provide theoretical guarantees on achievement probabilities and demonstrate substantial empirical gains in planning accuracy and help-rate reductions on AI2THOR-based tasks as team size grows. Overall, the approach offers a practical, scalable framework for reliable NL-based multi-robot planning with provable performance assurances.
Abstract
This paper addresses task planning problems for language-instructed robot teams. Tasks are expressed in natural language (NL), requiring the robots to apply their capabilities at various locations and semantic objects. Several recent works have addressed similar planning problems by leveraging pre-trained Large Language Models (LLMs) to design effective multi-robot plans. However, these approaches lack performance guarantees. To address this challenge, we introduce a new distributed LLM-based planner, called S-ATLAS for Safe plAnning for Teams of Language-instructed AgentS, that is capable of achieving user-defined mission success rates. This is accomplished by leveraging conformal prediction (CP), a distribution-free uncertainty quantification tool in black-box models. CP allows the proposed multi-robot planner to reason about its inherent uncertainty in a distributed fashion, enabling robots to make individual decisions when they are sufficiently certain and seek help otherwise. We show, both theoretically and empirically, that the proposed planner can achieve user-specified task success rates, assuming successful plan execution, while minimizing the overall number of help requests. We provide comparative experiments against related works showing that our method is significantly more computational efficient and achieves lower help rates. The advantage of our algorithm over baselines becomes more pronounced with increasing robot team size.