A System for Human-Robot Teaming through End-User Programming and Shared Autonomy

TL;DR

The paper tackles the challenge of automating variable, contact-rich industrial sanding tasks by coupling a human operator with a robot through end-user programming and shared autonomy. It presents a prototype system where a mobile-base cobot, equipped with an RGB-D sensor and a sanding tool, is programmed via a tablet AR interface and augmented by real-time operator corrections modeled as $\mathbf{x} = \mathbf{x}_n + \delta \mathbf{x}$. Two workflows—structured and unstructured—address different task knowledge levels, supported by a common back-end for reachability checks and corrections, demonstrated in lab and on-site aviation contexts. The study highlights ergonomic benefits and the need to balance speed and flexibility, outlining future work on scalable interfaces, automatic parameterization, and broader task applicability. The work advances flexible human-robot teaming for complex manufacturing tasks and informs design directions for industrial cobots in ergonomically demanding settings.

Abstract

Many industrial tasks-such as sanding, installing fasteners, and wire harnessing-are difficult to automate due to task complexity and variability. We instead investigate deploying robots in an assistive role for these tasks, where the robot assumes the physical task burden and the skilled worker provides both the high-level task planning and low-level feedback necessary to effectively complete the task. In this article, we describe the development of a system for flexible human-robot teaming that combines state-of-the-art methods in end-user programming and shared autonomy and its implementation in sanding applications. We demonstrate the use of the system in two types of sanding tasks, situated in aircraft manufacturing, that highlight two potential workflows within the human-robot teaming setup. We conclude by discussing challenges and opportunities in human-robot teaming identified during the development, application, and demonstration of our system.

Figures (6)

• Figure 1: In this paper, we describe the development of a human-robot teaming solution for variable industrial sanding tasks that combines the strengths of the operator and robot through end-user programming and shared autonomy.
• Figure 2: Proposed approach. The worker leverages a mobile augmented-reality interface and custom haptic device to program and augment a robot completing a task. The robot is on a mobile base and mounted with a RGB-D camera for localization and contextualized programming.
• Figure 3: Overview of implementation. The worker's interface uses a touchscreen gamepad and (optional) low degree-of-freedom haptic device to interact with the robot. The robot has a custom stand and end effector for sanding.
• Figure 4: Example interfaces for the structured and unstructured workflow. The structured workflow mostly leverages the scanned 3D view of the geometry. The unstructured workflow is programmed as an overlay on the current robot camera view. Both workflows include state indicators (e.g., robot state) and sandpaper monitoring.
• Figure 5: Workflow for structured tasks. The task is to sand white paint off a curved composite structure. The workpiece is large enough such that it requires multiple configurations of sanding to complete. Event timing (MM:SS) is reported in parentheses.