Integrating Retrospective Framework in Multi-Robot Collaboration

TL;DR

This paper addresses the challenge of efficient, adaptable decision-making in dynamic multi-robot environments by introducing a retrospective actor-critic framework that leverages two LLMs to coordinate joint actions and retrospectively refine plans. The actor generates actions from current observations $o_t^n$ and task $T$, while the critic analyzes outcomes using short-term memory $S$ and long-term memory $M_{i+1}$ to produce feedback that informs future prompts $p_{k+1}^n$. Across RoCoBench simulations with $L$LMs $LLM_1$ and $LLM_2$, the approach improves task success on several benchmarks (e.g., Arrange Cabinet, Sort Cubes, Move Rope) and reduces replanning, demonstrating enhanced adaptability and robustness, albeit with occasional hallucinations from un-tuned LLMs. The ablation studies show the importance of context length and model selection, guiding future tuning and deployment in real-world scenarios.

Abstract

Recent advancements in Large Language Models (LLMs) have demonstrated substantial capabilities in enhancing communication and coordination in multi-robot systems. However, existing methods often struggle to achieve efficient collaboration and decision-making in dynamic and uncertain environments, which are common in real-world multi-robot scenarios. To address these challenges, we propose a novel retrospective actor-critic framework for multi-robot collaboration. This framework integrates two key components: (1) an actor that performs real-time decision-making based on observations and task directives, and (2) a critic that retrospectively evaluates the outcomes to provide feedback for continuous refinement, such that the proposed framework can adapt effectively to dynamic conditions. Extensive experiments conducted in simulated environments validate the effectiveness of our approach, demonstrating significant improvements in task performance and adaptability. This work offers a robust solution to persistent challenges in robotic collaboration.

Figures (2)

• Figure 1: The illustration of a multi-agent collaborative task involving robotic arms (Agent A and Agent B in this case) is coordinated by two large language models, $LLM_1$ and $LLM_2$. In the first stage (collaborative discussion), agents collaboratively discuss an action plan, assigning tasks such as picking and placing cubes on specific panels. After validating the plan for inverse kinematics and collision avoidance, it is stored in short-term memory and passed to $LLM_2$ for retrospective discussion and planning. $LLM_2$ contains an action critic that provides a high-level overview for improving performance, while an action proposer offers more detailed suggestions. This feedback is stored in long-term memory. Finally, only two rounds of feedback are selected as the context for $LLM_1$ to produce an improved action plan, which is then executed in the environment.
• Figure 2: Simulation qualitative results. Here, we demonstrate three tasks completed by multiple agents. Each row represents the steps performed by multiple robots in cooperation for a given task.