Failure-Aware Bimanual Teleoperation via Conservative Value Guided Assistance

TL;DR

Experimental results on contact-rich manipulation tasks demonstrate improved task success rates and reduced operator workload compared to conventional teleoperation and shared-autonomy baselines, indicating that conservative value learning provides an effective mechanism for embedding failure awareness into bilateral teleoperation.

Abstract

Teleoperation of high-precision manipulation is con-strained by tight success tolerances and complex contact dy-namics, which make impending failures difficult for human operators to anticipate under partial observability. This paper proposes a value-guided, failure-aware framework for bimanual teleoperation that provides compliant haptic assistance while pre-serving continuous human authority. The framework is trained entirely from heterogeneous offline teleoperation data containing both successful and failed executions. Task feasibility is mod-eled as a conservative success score learned via Conservative Value Learning, yielding a risk-sensitive estimate that remains reliable under distribution shift. During online operation, the learned success score regulates the level of assistance, while a learned actor provides a corrective motion direction. Both are integrated through a joint-space impedance interface on the master side, yielding continuous guidance that steers the operator away from failure-prone actions without overriding intent. Experimental results on contact-rich manipulation tasks demonstrate improved task success rates and reduced operator workload compared to conventional teleoperation and shared-autonomy baselines, indicating that conservative value learning provides an effective mechanism for embedding failure awareness into bilateral teleoperation. Experimental videos are available at https://www.youtube.com/watch?v=XDTsvzEkDRE

Figures (6)

• Figure 1: Our failure-aware teleoperation framework leverages conservative success score learning to trigger corrective torque on the leader arm, guiding the operator away from out-of-distribution (OOD) behaviors.
• Figure 2: Overview of the proposed value-guided shared autonomy framework. Upper part: Offline learning from heterogeneous teleoperation data with binary success and failure labels to train a conservative success score (critic). Bottom part: Online deployment, where the success score determines when intervention is required, and a learned actor provides a corrective guidance action. Both are integrated within a shared autonomy framework to guide the operator away from out-of-distribution and failure-prone actions while preserving continuous human control.
• Figure 3: Value-Guided Impedance Assistance. Activated guidance generates compliant leader-side torques that discourage unsafe deviations while preserving continuous human authority.
• Figure 4: The snapshots of all ten experiments, including four tasks in Group 1 (basic manipulation tasks) with follower-view (in purple box) and leader-view (in pink box), as well as six tasks in Group 2 (interactive fine-grained manipulation tasks) with follower-view (in yellow box) and top-view (in green box).
• Figure 5: Comparison of completion time for all tasks under baselines and our method.