Interface-Aware Trajectory Reconstruction of Limited Demonstrations for Robot Learning

TL;DR

A trajectory reconstruction algorithm is presented that reasons about task, environment, and interface constraints to lift demonstrations into the robot's full control space and shows that lifted trajectories are faster and more efficient than their interface-constrained counterparts while respecting user preferences.

Abstract

Assistive robots offer agency to humans with severe motor impairments. Often, these users control high-DoF robots through low-dimensional interfaces, such as using a 1-D sip-and-puff interface to operate a 6-DoF robotic arm. This mismatch results in having access to only a subset of control dimensions at a given time, imposing unintended and artificial constraints on robot motion. As a result, interface-limited demonstrations embed suboptimal motions that reflect interface restrictions rather than user intent. To address this, we present a trajectory reconstruction algorithm that reasons about task, environment, and interface constraints to lift demonstrations into the robot's full control space. We evaluate our approach using real-world demonstrations of ADL-inspired tasks performed via a 2-D joystick and 1-D sip-and-puff control interface, teleoperating two distinct 7-DoF robotic arms. Analyses of the reconstructed demonstrations and derived control policies show that lifted trajectories are faster and more efficient than their interface-constrained counterparts while respecting user preferences.

Figures (8)

• Figure 1: Limited interfaces and control modes. Sip/Puff Breeze™ with Headset (left). Image © Origin Instruments OriginInstruments, reproduced with permission. 2-D joystick (right). An example of modal partitioning of the robot control space (bottom, end-effector translation $\vec{v}$ and rotation $\vec{\omega}$ velocity vectors, and gripper $g$ open/close.)
• Figure 2: Illustrative example comparing raw, smoothed, and reconstructed trajectories from two different interfaces. Left: The demonstrator intentionally issues lower-dimensional (1-D) commands through a higher-dimensional (2-D) interface. Right: The demonstration instead is constrained by a lower-dimensional (1-D) interface.
• Figure 3: Experimental setup of each of the tasks. The xArm7 robotic arm with a pincher gripper is pictured in the same initial condition for each task.
• Figure 4: Percentage of dimensions activated across the full trajectory, averaged over all demonstrations. Raw (top row) and reconstructed (bottom row) trajectories collected with the joystick (left column) and sip/puff (right column.
• Figure 5: Evolution of dimension activations using the 2-D joystick for the Pick-and-place task. Activations in the raw demonstration (top) are 1-D or 2-D only, while the reconstructed demonstration (bottom) lifts as high as the full 6-D.