CoBT: Collaborative Programming of Behaviour Trees from One Demonstration for Robot Manipulation

TL;DR

CoBT tackles agile robot programming under variability by learning a complete task policy as a deployable BT from a single demonstration, integrating automatic segmentation, DMP-based motion primitives, and logic-based declarative learning for BT generation and goal adaptation. It delivers reactive, modular behavior trees capable of reusing learned primitives to handle long-horizon tasks, validated on seven manipulation tasks with high success rates and fast programming times. A pilot study with non-experts indicates good usability and low cognitive load. The results suggest CoBT can enable rapid, human-in-the-loop robot programming in industrial settings, though perception accuracy and DMP generalization remain key limitations to address.

Abstract

Mass customization and shorter manufacturing cycles are becoming more important among small and medium-sized companies. However, classical industrial robots struggle to cope with product variation and dynamic environments. In this paper, we present CoBT, a collaborative programming by demonstration framework for generating reactive and modular behavior trees. CoBT relies on a single demonstration and a combination of data-driven machine learning methods with logic-based declarative learning to learn a task, thus eliminating the need for programming expertise or long development times. The proposed framework is experimentally validated on 7 manipulation tasks and we show that CoBT achieves approx. 93% success rate overall with an average of 7.5s programming time. We conduct a pilot study with non-expert users to provide feedback regarding the usability of CoBT.

Figures (6)

• Figure 1: CoBT: a collaborative programming framework that generates deployable behavior trees from one demonstration. This example shows the work-flow of programming opening a drawer task.
• Figure 2: A high-level workflow of offline learning and execution in CoBT. The red lines, depict the task (BT) and action (DMP) policies that are learned offline through PbD. During execution, depicted with black lines, the user-defined goal is abstracted to adapt the BT parameters. The generated BT decides "what" action to execute, and DMPs control "how" the action should be executed based on the current environmental conditions.
• Figure 3: Atomic BT. The effect-conditions $c^{i}_{eff}$ are achieved by action $Action^i$. To execute the action, pre-condition $c^i_{pre}$ should satisfy, The BT starts with a fallback node connected to a parallel node for effect-conditions and a sequence node for action and pre-conditions.
• Figure 4: 7 evaluation tasks that include mix of complex and P2P trajectory executions, and short and long-horizon tasks with multi-level goals.
• Figure 5: (Top) Segmentation based on velocity and gripper state. (Middle) Transitions during the drawer task demonstration. For example, the state transition from a to b (middle) occurs due to action in segment 1 (top). (Bottom) a trial example of the generated policy under normal conditions.