When a Robot is More Capable than a Human: Learning from Constrained Demonstrators

TL;DR

This work addresses learning from constrained demonstrations where the expert’s interface limits actions, by proposing LfCD-GRIP, which learns a state-only goal-proximity reward and propagates it through confident online observations via trajectory-wise interpolation. The approach decouples reward learning from constrained actions, uses Monte Carlo Dropout to estimate confidence, and gradually interpolates proximities along agent rollouts to guide exploration toward efficient trajectories. Empirical results across navigation, manipulation, and real-robot WidowX tasks show that LfCD-GRIP achieves shorter, faster trajectories than imitation-learning baselines and can leverage actions outside the expert’s constraint, including significantly faster real-robot performance (e.g., 12s vs 100s in WidowX-Pick). Overall, the method demonstrates a practical path to surpass constrained demonstrations by learning a generalizable progress signal and safely propagating it to unseen states, improving sample efficiency and task completion time.

Abstract

Learning from demonstrations enables experts to teach robots complex tasks using interfaces such as kinesthetic teaching, joystick control, and sim-to-real transfer. However, these interfaces often constrain the expert's ability to demonstrate optimal behavior due to indirect control, setup restrictions, and hardware safety. For example, a joystick can move a robotic arm only in a 2D plane, even though the robot operates in a higher-dimensional space. As a result, the demonstrations collected by constrained experts lead to suboptimal performance of the learned policies. This raises a key question: Can a robot learn a better policy than the one demonstrated by a constrained expert? We address this by allowing the agent to go beyond direct imitation of expert actions and explore shorter and more efficient trajectories. We use the demonstrations to infer a state-only reward signal that measures task progress, and self-label reward for unknown states using temporal interpolation. Our approach outperforms common imitation learning in both sample efficiency and task completion time. On a real WidowX robotic arm, it completes the task in 12 seconds, 10x faster than behavioral cloning, as shown in real-robot videos on https://sites.google.com/view/constrainedexpert .

Figures (11)

• Figure 1: A human expert constrained by a mode-switching joystick produces segmented trajectories. A robot employing LfCD-GRIP executes smooth and efficient motions beyond the demonstrations.
• Figure 1: Success rate and OOC action ratio in Maze2D-Constrained. LfCD-GRIP achieves 100% success while effectively leveraging OOC actions.
• Figure 2: Proximity is interpolated between high-confidence anchors.
• Figure 3: We use various manipulation and navigation tasks with different kinds and degrees of constrained expert demonstration datasets.
• Figure 4: MiniGrid-LfCD Results. (left) The expert follows the blue path to the green goal, while LfCD-GRIP takes the red shortcut; (right) average episode length across methods.