Learning Task Specifications from Demonstrations as Probabilistic Automata

TL;DR

Problem: robustly specifying long-horizon robotic tasks from unstructured demonstrations is challenging for traditional LfD approaches. Approach: extract frequent sub-goals, build a Probabilistic Deterministic Finite Automaton (PDFA) that encodes valid sub-goal sequences and demonstrator preferences, and use it for planning with a low-level controller. Contributions: an efficient pipeline for sub-goal inference via clustering, PDFA construction from demonstration traces, and planning that adapts to missing or changing conditions; validated on object manipulation with a real robot and on simulated drone and arm tasks. Impact: provides an interpretable, online-planning task specification that accommodates diverse demonstrations and long-horizon tasks.

Abstract

Specifying tasks for robotic systems traditionally requires coding expertise, deep domain knowledge, and significant time investment. While learning from demonstration offers a promising alternative, existing methods often struggle with tasks of longer horizons. To address this limitation, we introduce a computationally efficient approach for learning probabilistic deterministic finite automata (PDFA) that capture task structures and expert preferences directly from demonstrations. Our approach infers sub-goals and their temporal dependencies, producing an interpretable task specification that domain experts can easily understand and adjust. We validate our method through experiments involving object manipulation tasks, showcasing how our method enables a robot arm to effectively replicate diverse expert strategies while adapting to changing conditions.

Figures (4)

• Figure 1: 1) Given a set of demonstrations, potential sub-goals are extracted through clustering. 2) From the set of sub-goals and the demonstrations a PDFA is constructed representing task structure and demonstrator preferences. 3) The learned PDFA can be used for task planning.
• Figure 2: Left: Inferred PDFA (from 9 demonstrations) for a task involving the construction of two stacks of two blocks, with each block required to be placed in a designated location. Top right: The initial plan selects sub-goals based on the highest transition probabilities. However, during execution, the unavailability of the yellow block prevents the agent from completing sub-goal 2, necessitating two re-planning steps. Bottom right: Snapshots of the environment during task execution. The top row displays frames captured by the camera mounted on the robot's end effector, while the bottom row shows side-view frames captured by an additional camera.
• Figure 3: Effect of different task properties on the cluster and PDFA inference time. The investigated properties are the number of demonstrations $\mid \Omega \mid$ (Fig. \ref{['fig:result_a']} and \ref{['fig:result_e']}), the number of sub-goals $\mid \Sigma \mid$ (Fig. \ref{['fig:result_b']} and \ref{['fig:result_f']}), the language size $\mathcal{L}(\mathcal{A})$ (Fig. \ref{['fig:result_c']} and \ref{['fig:result_g']}) and the number of objects $I$ (Fig. \ref{['fig:result_d']} and \ref{['fig:result_h']}).
• Figure 4: (a) drone surveillance (b) two-jointed robot arm.

Theorems & Definitions (2)

• Definition III.1
• Definition III.2