Versatile Demonstration Interface: Toward More Flexible Robot Demonstration Collection

TL;DR

The paper addresses the challenge that Learning from Demonstration (LfD) systems benefit from multiple demonstration modalities, but existing tools are not generally multi-modal. It introduces the Versatile Demonstration Interface (VDI), a robot-mounted tool that supports teleoperation, kinesthetic teaching, and natural demonstrations using onboard sensing (camera, force sensor, and AprilTag-based tracking) and a camera-pose optimization for natural demonstrations. A UR5e-based prototype showcases VDI's capabilities, accompanied by a user study with manufacturing experts to evaluate modality utility, switching, and practical use cases in representative tasks. Results indicate natural demonstrations are often preferred and that VDI provides valuable flexibility for manufacturing tasks, offering design insights for data collection and future integration with robot policy learning.

Abstract

Previous methods for Learning from Demonstration leverage several approaches for a human to teach motions to a robot, including teleoperation, kinesthetic teaching, and natural demonstrations. However, little previous work has explored more general interfaces that allow for multiple demonstration types. Given the varied preferences of human demonstrators and task characteristics, a flexible tool that enables multiple demonstration types could be crucial for broader robot skill training. In this work, we propose Versatile Demonstration Interface (VDI), an attachment for collaborative robots that simplifies the collection of three common types of demonstrations. Designed for flexible deployment in industrial settings, our tool requires no additional instrumentation of the environment. Our prototype interface captures human demonstrations through a combination of vision, force sensing, and state tracking (e.g., through the robot proprioception or AprilTag tracking). Through a user study where we deployed our prototype VDI at a local manufacturing innovation center with manufacturing experts, we demonstrated VDI in representative industrial tasks. Interactions from our study highlight the practical value of VDI's varied demonstration types, expose a range of industrial use cases for VDI, and provide insights for future tool design.

Figures (6)

• Figure 1: Versatile Demonstration Interface is a tool that connects to the end of a collaborative robot and makes it easier to collect demonstration data through teleoperation, kinesthetic teaching, and natural demonstrations (where the tool detaches from and is tracked by the robot).
• Figure 2: Examples of the three demonstration modes of MMDI in a rolling task. (Left) The operator remotely controls the robot through a 6D input device. (Middle) The operator directly manipulates the robot arm and the environment forces are measured through a distal force sensor. (Right) The operator detaches the tool and performs demonstrations naturally while the robot optimizes its pose to track the tool with the interface's camera.
• Figure 3: Schematic of the prototype interface. The interface consists of two main sections: the robot mount and the human interface. The schematic also illustrates the key features supporting the interaction modes.
• Figure 4: User evaluation tasks. Left: The press-fitting task where participants press sixteen fittings into designated slots. The two groups of fittings were on opposite ends of the workbench. Right: Rolling task where participants used the end effector to try and evenly distribute the material over the contoured surface. A pin helps to keep the material in place.
• Figure 5: User study questionnaire results comparing the three demonstration modalities in terms of task load, usability, usefulness, and ease of use.