Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Autonomous Integration and Improvement of Robotic Assembly using Skill Graph Representations

Peiqi Yu, Philip Huang, Chaitanya Chawla, Guanya Shi, Jiaoyang Li, Changliu Liu

Abstract

Robotic assembly systems traditionally require substantial manual engineering effort to integrate new tasks, adapt to new environments, and improve performance over time. This paper presents a framework for autonomous integration and continuous improvement of robotic assembly systems based on Skill Graph representations. A Skill Graph organizes robot capabilities as verb-based skills, explicitly linking semantic descriptions (verbs and nouns) with executable policies, pre-conditions, post-conditions, and evaluators. We show how Skill Graphs enable rapid system integration by supporting semantic-level planning over skills, while simultaneously grounding execution through well-defined interfaces to robot controllers and perception modules. After initial deployment, the same Skill Graph structure supports systematic data collection and closed-loop performance improvement, enabling iterative refinement of skills and their composition. We demonstrate how this approach unifies system configuration, execution, evaluation, and learning within a single representation, providing a scalable pathway toward adaptive and reusable robotic assembly systems. The code is at https://github.com/intelligent-control-lab/AIDF.

Autonomous Integration and Improvement of Robotic Assembly using Skill Graph Representations

Abstract

Robotic assembly systems traditionally require substantial manual engineering effort to integrate new tasks, adapt to new environments, and improve performance over time. This paper presents a framework for autonomous integration and continuous improvement of robotic assembly systems based on Skill Graph representations. A Skill Graph organizes robot capabilities as verb-based skills, explicitly linking semantic descriptions (verbs and nouns) with executable policies, pre-conditions, post-conditions, and evaluators. We show how Skill Graphs enable rapid system integration by supporting semantic-level planning over skills, while simultaneously grounding execution through well-defined interfaces to robot controllers and perception modules. After initial deployment, the same Skill Graph structure supports systematic data collection and closed-loop performance improvement, enabling iterative refinement of skills and their composition. We demonstrate how this approach unifies system configuration, execution, evaluation, and learning within a single representation, providing a scalable pathway toward adaptive and reusable robotic assembly systems. The code is at https://github.com/intelligent-control-lab/AIDF.
Paper Structure (29 sections, 1 equation, 8 figures, 1 table)

This paper contains 29 sections, 1 equation, 8 figures, 1 table.

Table of Contents

  1. Introduction
  2. Related Work
  3. Task and Skill Representations in Robotics
  4. Learning-Based Manipulation and Assembly
  5. Skill-Centric and Neuro-Symbolic Frameworks
  6. Skill Graph Representation
  7. Skill Definition
  8. Skill Graph Structure
  9. Atomic and Meta Skills
  10. Autonomous System Integration Using Skill Graphs
  11. Semantic-Level Planning
  12. Execution-Level Grounding
  13. Intuitive Task Specification via Video Extraction
  14. Data Collection and Performance Evaluation
  15. Skill-Level Logging
  16. ...and 14 more sections

Figures (8)

  • Figure 1: Overview of the Skill Graph representation and its integration with a bimanual robotic LEGO assembly task.
  • Figure 2: Intuitive Task Specification via Video Extraction. The pipeline transforms a raw human demonstration video into a structured Skill Graph.
  • Figure 3: Skill-level trajectory visualization. The upper panels present representative execution snapshots for different skills (Pick, Place Up, Place Down, Support Top/Bottom, Handover, and Transit) performed by the two Yaskawa GP4 robots (named as "DESTROYER" and "ARCHITECT"). The lower plots illustrate the temporal evolution of two robots' joint position and force over the full task duration separated by different skill labels, illustrating motion dynamics throughout the entire manipulation sequence with skill labels.
  • Figure 4: Intuitive Task Specification Results: Transfer from Human demonstration to Robot Execution.
  • Figure 5: Using planning and execution data to craft new vision-based perception in Skill Graph. In the top, we present the sources of data captured during skill-based planning and execution of a LEGO assembly sequence. The bottom half shows a pick/place post-condition evaluator from an in-hand camera and anomaly detection skill from side-view cameras.
  • ...and 3 more figures