Real-Time Operator Takeover for Visuomotor Diffusion Policy Training

TL;DR

The paper addresses robustness gaps in imitation learning for visuomotor diffusion policies by introducing Real-Time Operator Takeover (RTOT), which lets a human operator intervene in real time to correct undesirable states and collects only the takeover trajectories for subsequent retraining. The approach interleaves initial demonstrations with targeted takeover demonstrations, retraining the policy iteratively to cover failure modes while reducing total demonstration time. A Mahalanobis distance analysis on image and pose embeddings investigates out-of-distribution detection as a diagnostic tool for deployment, revealing that high distances correlate with failure states but do not strictly predict final task performance. Experimental results on a cyclic rice scooping task show that RTOT-produced policies outperform baselines trained on the same or greater amounts of initial data, confirming improved data efficiency and robustness, and demonstrating practical benefits for real-world robotic manipulation.

Abstract

We present a Real-Time Operator Takeover (RTOT) paradigm enabling operators to seamlessly take control of a live visuomotor diffusion policy, guiding the system back into desirable states or reinforcing specific demonstrations. We present new insights in using the Mahalonobis distance to automatically identify undesirable states. Once the operator has intervened and redirected the system, the control is seamlessly returned to the policy, which resumes generating actions until further intervention is required. We demonstrate that incorporating the targeted takeover demonstrations significantly improves policy performance compared to training solely with an equivalent number of, but longer, initial demonstrations. We provide an in-depth analysis of using the Mahalanobis distance to detect out-of-distribution states, illustrating its utility for identifying critical failure points during execution. Supporting materials, including videos of initial and takeover demonstrations and all rice scooping experiments, are available on the project website: https://operator-takeover.github.io/

Figures (7)

• Figure 1: Real-Time Operator Takeover paradigm: after training a policy with a small number of initial demonstrations, we run the policy in the environment with the operator on standby. As soon as the policy enters an undesirable state, the operator seamlessly takes over, and only the takeover portion is recorded as new demonstrations. A new policy is trained, and the paradigm can be repeated until the desired performance is achieved.
• Figure 2: Real-Time Operator Takeover in action: The policy $\pi_I$ controls the robot until a state is reached where the operator must take over to avoid spilling rice on the table. After the operator completes the intervention (depositing the rice in the bowl), control is seamlessly returned to $\pi_I$.
• Figure 3: Illustration of the Real-Time Takeover process: While the policy sends actions, observations (end-effector RGB view and robot pose) are continuously stored in a ring buffer. When the operator takes control using the VR controller, these ring buffer observations, along with subsequent data, are recorded as a new demonstration in $\mathcal{D}_T$.
• Figure 4: Demonstration lengths for the datasets used to train the evaluated visuomotor policies. The takeover paradigm produces significantly shorter demonstrations on average, highlighting its efficiency.
• Figure 5: Detailed results of the cyclic rice scooping experiments. The amount of rice (in grams) is shown for each of the $10$ trials across all five evaluated policies.