Visualizing Impedance Control in Augmented Reality for Teleoperation: Design and User Evaluation

Abstract

Teleoperation for contact-rich manipulation remains challenging, especially when using low-cost, motion-only interfaces that provide no haptic feedback. Virtual reality controllers enable intuitive motion control but do not allow operators to directly perceive or regulate contact forces, limiting task performance. To address this, we propose an augmented reality (AR) visualization of the impedance controller's target pose and its displacement from each robot end effector. This visualization conveys the forces generated by the controller, providing operators with intuitive, real-time feedback without expensive haptic hardware. We evaluate the design in a dual-arm manipulation study with 17 participants who repeatedly reposition a box with and without the AR visualization. Results show that AR visualization reduces completion time by 24% for force-critical lifting tasks, with no significant effect on sliding tasks where precise force control is less critical. These findings indicate that making the impedance target visible through AR is a viable approach to improve human-robot interaction for contact-rich teleoperation.

Figures (5)

• Figure 1: Example of contact-rich teleoperation. A human operator collaborates with two robot arms to lift a box; a task requiring careful regulation of contact forces. The operator uses VR motion-tracking controllers with an impedance controller to command the end effectors. Our proposed AR visualization displays the impedance controller's target pose (blue disks) and the position tracking offset (blue lines), providing intuitive visual feedback that correlates with the generated forces. A video of participants operating the system is available at https://youtu.be/WQA8USq9lXY.
• Figure 2: Schematic drawing of a robot arm (grey), the current end-effector pose (red), the target pose (blue) and the handheld VR controller (green) with their corresponding coordinate frames. The target pose is a digital entity and cannot be seen by the naked eye.
• Figure 3: Design of our impedance visualization shown from the perspective of an operator lifting an object. The blue disks represent the 6-DoF target poses of the end effectors, while the blue lines indicate target offset in translation, which correlate with the virtual forces applied by the controller. Four consecutive snapshots show the task progression: (a) no contact, (b) initial contact, (c) clamping the box, and (d) lifting it. Without this visualization, users cannot directly perceive the difference between (b) and (c), making force regulation difficult.
• Figure 4: Examples of the manipulation tasks: sliding (top) and lifting (bottom). The green cuboid, displayed in augmented reality, indicates the target location for the box. Images on the left show when the target first appeared, while images on the right show the moment of target completion.
• Figure 5: Target completion times for the lifting and sliding tasks, estimated from a linear mixed-effects model fitted on participant data. Reported values represent EMMs for both visualization conditions and the contrast, back-transformed from the model's log scale.

Theorems & Definitions (3)

• Definition 1: Impedance controller
• Definition 2: Hand-target mapping
• Definition 3: Target Completion Criteria