LLM-BT: Performing Robotic Adaptive Tasks based on Large Language Models and Behavior Trees

TL;DR

This work addresses the challenge of enabling autonomous robots to adapt to external disturbances by combining large language models with Behavior Trees. The proposed LLM-BT pipeline uses a Recognition module to build semantic maps, Reasoning with ChatGPT to generate task steps, a BERT-based Parser to construct an initial BT from keywords, and a BTs Update mechanism to dynamically expand and insert new actions as needed. Key contributions include (1) automatic construction of variable BTs via LLMs, (2) a BTs Update algorithm that expands subtrees to satisfy failed conditions, and (3) empirical validation in cargo sorting and household service scenarios demonstrating robustness to disturbances and explicit discussion of advantages and limitations. The results indicate practical promise for adaptive robotic tasks while highlighting areas for improvement in scene understanding and ATL template design, with future work focusing on parsing accuracy and deformable-object manipulation.

Abstract

Large Language Models (LLMs) have been widely utilized to perform complex robotic tasks. However, handling external disturbances during tasks is still an open challenge. This paper proposes a novel method to achieve robotic adaptive tasks based on LLMs and Behavior Trees (BTs). It utilizes ChatGPT to reason the descriptive steps of tasks. In order to enable ChatGPT to understand the environment, semantic maps are constructed by an object recognition algorithm. Then, we design a Parser module based on Bidirectional Encoder Representations from Transformers (BERT) to parse these steps into initial BTs. Subsequently, a BTs Update algorithm is proposed to expand the initial BTs dynamically to control robots to perform adaptive tasks. Different from other LLM-based methods for complex robotic tasks, our method outputs variable BTs that can add and execute new actions according to environmental changes, which is robust to external disturbances. Our method is validated with simulation in different practical scenarios.

Figures (11)

• Figure 1: Overview of LLM-BT.
• Figure 2: Pipeline of Recognition.
• Figure 3: Pipeline of Reasoning.
• Figure 4: Pipeline of keywords extraction.
• Figure 5: The process of keywords parsing.