Hierarchical Task Decomposition for Execution Monitoring and Error Recovery: Understanding the Rationale Behind Task Demonstrations

TL;DR

The paper tackles the challenge of learning robust, in-contact robotic manipulation by introducing a hierarchical, incremental framework that learns low-level motion primitives and high-level task structure from real demonstrations. It combines Bayesian nonparametric Gaussian Inverse Reinforcement Learning (BN-GIRL) for unsupervised task segmentation with probabilistic feature clustering to form a Task Graph of skills, each with subgoals and feature constraints. Autonomous execution relies on Gaussian Mixture Regression-based motion generation and a two-tier anomaly-detection scheme that differentiates epistemic and aleatoric uncertainty, enabling recovery behaviors to be learned online and appended to the Task Graph. The approach is validated on simulation and two real robots across multiple tasks, demonstrating data-efficient learning (as few as three demonstrations), competitive segmentation accuracy, and robust anomaly detection and recovery; it significantly reduces training data and computational demands relative to state-of-the-art methods. Overall, the framework offers a practical, scalable route to data-efficient, continual learning of complex contact tasks with online monitoring and recovery capabilities.

Abstract

Multi-step manipulation tasks where robots interact with their environment and must apply process forces based on the perceived situation remain challenging to learn and prone to execution errors. Accurately simulating these tasks is also difficult. Hence, it is crucial for robust task performance to learn how to coordinate end-effector pose and applied force, monitor execution, and react to deviations. To address these challenges, we propose a learning approach that directly infers both low- and high-level task representations from user demonstrations on the real system. We developed an unsupervised task segmentation algorithm that combines intention recognition and feature clustering to infer the skills of a task. We leverage the inferred characteristic features of each skill in a novel unsupervised anomaly detection approach to identify deviations from the intended task execution. Together, these components form a comprehensive framework capable of incrementally learning task decisions and new behaviors as new situations arise. Compared to state-of-the-art learning techniques, our approach significantly reduces the required amount of training data and computational complexity while efficiently learning complex in-contact behaviors and recovery strategies. Our proposed task segmentation and anomaly detection approaches outperform state-of-the-art methods on force-based tasks evaluated on two different robotic systems.

Figures (18)

• Figure 1: The different phases and challenges of the box grasping and locking task. The upper row shows EEF configurations in the subgoal region of the respective skills during the execution which trigger a transition to the successor skills. The bottom row highlights the difficulties during each skill. In the first skill, the lower part of the gripper must not collide with the box, while the movable slides are positioned above the box. In the next phases, the robot must apply force via the front part of the slides, without pushing the locking pin inside the slides, and maintain contact with the side wall of the box until the third subgoal configuration is reached. After that, the locking pin must be pushed to compress the springs in the slides while rotating the gripper into the vertical configuration. If there is not enough force exerted to push the gripper down, the tight clearance between the box and the box locks on the lower part of the gripper causes a collision. A video of the task learning and execution can be seen in Extension 3.
• Figure 2: Our proposed incremental high- and low-level task learning framework. The process of learning a new task starts with an initial teaching sequence ① (Sec. \ref{['sec:initial-ts']}), where the user provides several demonstrations of the intended task that are segmented using BNG-IRL to learn an initial task model. The low-level skills of this initial model can then be further refined during the skill refinement teaching sequence ② (Sec. \ref{['sec:skill-refinement-ts']}), depicted at the bottom. During this phase, the robot performs the initial task, while the user supervises and assists in phases, where the skills need refinement. The recorded training data is used to update the skills. The last teaching sequence ③ (Sec. \ref{['sec:task-decision-ts']}) is triggered if the execution module identifies a new anomaly (see Sec. \ref{['sec:motion-generation-anomaly-detection']} and \ref{['sec:skill-selection']}). Similar to the initial teaching sequence, the user provides a demonstration that shows how to recover from the anomaly, which is segmented and appended to the skill during which the anomaly occurred.
• Figure 3: Model and parameter inference of our proposed unsupervised task segmentation approach BNG-IRL.
• Figure 4: Simplified 2D example of subgoal-driven intention recognition based on IRL for a single demonstration.
• Figure 5: Steps of the box pushing task demonstration.