Geometrically-Aware One-Shot Skill Transfer of Category-Level Objects

TL;DR

The paper tackles generalizing manipulation skills to unseen category-level objects from a single demonstration. It combines Functional Maps (FM) to transfer object-centric interaction functions (RIF/EIF) with Task-Space Imitation Algorithm (TSIA) to produce geometry-aware trajectories, preserving demonstrated constraints in new scenes. Key contributions include a one-shot FM-based transfer framework, an online object-matching strategy using diagonal-dominance of the functional map, and TSIA-driven trajectory replanning, validated in diverse real-world scenes with minimal training. The findings indicate robust skill transfer with high success rates and efficient computation, enabling rapid adaptation to new objects without extensive retraining. This approach has practical potential for flexible, category-level manipulation in unstructured environments.

Abstract

Robotic manipulation of unfamiliar objects in new environments is challenging and requires extensive training or laborious pre-programming. We propose a new skill transfer framework, which enables a robot to transfer complex object manipulation skills and constraints from a single human demonstration. Our approach addresses the challenge of skill acquisition and task execution by deriving geometric representations from demonstrations focusing on object-centric interactions. By leveraging the Functional Maps (FM) framework, we efficiently map interaction functions between objects and their environments, allowing the robot to replicate task operations across objects of similar topologies or categories, even when they have significantly different shapes. Additionally, our method incorporates a Task-Space Imitation Algorithm (TSIA) which generates smooth, geometrically-aware robot paths to ensure the transferred skills adhere to the demonstrated task constraints. We validate the effectiveness and adaptability of our approach through extensive experiments, demonstrating successful skill transfer and task execution in diverse real-world environments without requiring additional training.

Figures (6)

• Figure 1: (a) User demonstrates the bottle shaking operation, and (b) robot imitates it on a different bottle using the proposed skill transfer framework.
• Figure 2: Pipeline of the proposed method, comprising the demonstration and imitation stages. The bottle stirring skill is considered here. In the demonstration stage, kinesthetic demonstrations are performed to capture the gripper's placement and object manipulation motions. The method then generates functions based on these demonstrations. When the robot encounters a new scene, these functions are transferred to similar objects, allowing the robot to determine corresponding gripper poses. With a new set of waypoints and the demonstrated trajectory, the tasks are generalised to novel scenes.
• Figure 3: Dataset of objects used for experiments, with objects marked with numbers are used for demonstration. Two user demonstrations are shown.
• Figure 4: Illustration of object matching in two multi-object scenes. For each scene object, a map is computed and shown on the right with scores overlaid. The object with the highest matching score is selected.
• Figure 5: Illustration of function transfer from the demonstration objects (top row) to other objects within the same category.

Theorems & Definitions (1)

• Definition : Skill