Attentiveness Map Estimation for Haptic Teleoperation of Mobile Robot Obstacle Avoidance and Approach

TL;DR

This work tackles unsafe or distracting haptic feedback in teleoperation by introducing Attentiveness Map Estimation (AME), a perception-driven mechanism that predicts operator attention to obstacles in real time. AME combines saliency (via Itti bottom-up models), a top-down mapping of visible obstacles, and a TBRS-inspired memory module to produce an attentiveness map, which modulates haptic feedback. The proposed Attentiveness-Modulated Generalized Potential Field (AMGPF) reduces repulsion for obstacles the operator is estimated to notice, enabling intentional approach while maintaining safety for unseen or forgotten obstacles. Experimental results in simulation show AMGPF improves task performance and reduces cognitive load relative to baselines, suggesting practical benefits for safer and more fluid haptic teleoperation in cluttered environments. Future work includes incorporating eye-tracking data and explicit task-intent estimation to further improve attentiveness accuracy.

Abstract

Haptic feedback can improve safety of teleoperated robots when situational awareness is limited or operators are inattentive. Standard potential field approaches increase haptic resistance as an obstacle is approached, which is desirable when the operator is unaware of the obstacle but undesirable when the movement is intentional, such as when the operator wishes to inspect or manipulate an object. This paper presents a novel haptic teleoperation framework that estimates the operator's attentiveness to obstacles and dampens haptic feedback for intentional movement. A biologically-inspired attention model is developed based on computational working memory theories to integrate visual saliency estimation with spatial mapping. The attentiveness map is generated in real-time, and our system renders lower haptic forces for obstacles that the operator is estimated to be aware of. Experimental results in simulation show that the proposed framework outperforms haptic teleoperation without attentiveness estimation in terms of task performance, robot safety, and user experience.

Figures (6)

• Figure 1: Diagram of the AME model. Saliency Module uses the RGB-D image to generate a saliency map. Mapping Module computes a top-down occupancy map and pairs currently visible points with saliency scores. Visible obstacles enter Memory Encoding and out-of-view obstacles undergo Memory Decay in the Memory Module, which updates the attentiveness map. Timesteps are exaggerated for illustration.
• Figure 2: Attentiveness map updates while approaching a wall. Attentiveness increases for obstacles in view and decays for out-of-view obstacles. Brighter color represents obstacles with higher attentiveness, leading to a reduced repulsive force.
• Figure 3: Estimated attentiveness modulates haptic feedback. Left: the operator's attentiveness is high around obstacle 1, resulting in a stronger obstacle 2 repulsion $f_{rep}^{2}$ due to lack of awareness on obstacle 2. Right: the operator's attentiveness is high around obstacle 2, resulting in a stronger obstacle 1 repulsion $f_{rep}^{1}$.
• Figure 4: Haptic feedback forces, indicated by red arrows, of AMGPF (ours) vs standard GPF along an evaluation trajectory. The trajectory is split into approach (left) and leaving (right) segments. The higher estimated attentiveness to the target location during approach causes AMGPF to reduce repulsive forces. (Best viewed in color)
• Figure 5: Simulated SATYRR robot and experiment setup