Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Design and Evaluation of an Assisted Programming Interface for Behavior Trees in Robotics

Jonathan Styrud, Matteo Iovino, Rebecca Stower, Mart Kartašev, Mikael Norrlöf, Mårten Björkman, Christian Smith

TL;DR

BEhavior TRee GUI (BETR-GUI) for creating BTs with the help of an AI assistant that combines methods using large language models, planning, genetic programming, and Bayesian optimization with a drag-and-drop editor is introduced.

Abstract

The possibility to create reactive robot programs faster without the need for extensively trained programmers is becoming increasingly important. So far, it has not been explored how various techniques for creating Behavior Tree (BT) program representations could be combined with complete graphical user interfaces (GUIs) to allow a human user to validate and edit trees suggested by automated methods. In this paper, we introduce BEhavior TRee GUI (BETR-GUI) for creating BTs with the help of an AI assistant that combines methods using large language models, planning, genetic programming, and Bayesian optimization with a drag-and-drop editor. A user study with 60 participants shows that by combining different assistive methods, BETR-GUI enables users to perform better at solving the robot programming tasks. The results also show that humans using the full variant of BETR-GUI perform better than the AI assistant running on its own.

Design and Evaluation of an Assisted Programming Interface for Behavior Trees in Robotics

TL;DR

BEhavior TRee GUI (BETR-GUI) for creating BTs with the help of an AI assistant that combines methods using large language models, planning, genetic programming, and Bayesian optimization with a drag-and-drop editor is introduced.

Abstract

The possibility to create reactive robot programs faster without the need for extensively trained programmers is becoming increasingly important. So far, it has not been explored how various techniques for creating Behavior Tree (BT) program representations could be combined with complete graphical user interfaces (GUIs) to allow a human user to validate and edit trees suggested by automated methods. In this paper, we introduce BEhavior TRee GUI (BETR-GUI) for creating BTs with the help of an AI assistant that combines methods using large language models, planning, genetic programming, and Bayesian optimization with a drag-and-drop editor. A user study with 60 participants shows that by combining different assistive methods, BETR-GUI enables users to perform better at solving the robot programming tasks. The results also show that humans using the full variant of BETR-GUI perform better than the AI assistant running on its own.
Paper Structure (36 sections, 4 equations, 10 figures, 9 tables)

This paper contains 36 sections, 4 equations, 10 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Background and related work
  3. Behavior Trees
  4. Planning
  5. Genetic Programming
  6. Bayesian Optimization
  7. LLMs
  8. Combinations
  9. Interface Design
  10. Instructions Tab
  11. Goal Editor Tab
  12. Behavior Tree Editor Tab
  13. AI Assistant
  14. Simulation Window
  15. Task Design
  16. ...and 21 more sections

Figures (10)

  • Figure 1: Example of a non-optimal behavior tree for solving the Cubes and bowl task. The different node types are described in Section \ref{['section:task_design']}.
  • Figure 2: Graphic representation showing an overview of the algorithms of BETR-GUI and their connections. Green boxes denote algorithms, and blue boxes denote data.
  • Figure 3: Screenshot of the Instructions tab of the GUI with the Tableware scenario description.
  • Figure 4: Screenshot of the Goal Editor tab of the GUI with the Cubes and bowl task with the goal conditions currently solving part of the task, as the goal condition of the blue cube is not yet correctly defined.
  • Figure 5: Screenshot of the Behavior Tree Editor tab of the GUI with the user's current tree to the left and the AI assistant's suggestion to the right. A hint to the user is shown in the textbox with a yellow background.
  • ...and 5 more figures