Gaze-Guided Robotic Vascular Ultrasound Leveraging Human Intention Estimation

TL;DR

The paper proposes a gaze-guided robotic ultrasound system to address operator variability in vascular imaging by tracking user gaze to select target vessels and by feeding this intention into a gaze-guided segmentation network. A Human Intention Estimation Module stabilizes noisy gaze signals and outputs an attention heatmap to guide segmentation, while a confidence-based orientation correction enhances probe-surface contact. Empirical results show superior segmentation performance over state-of-the-art methods, robustness of the attention estimation under gaze noise, and successful vessel switching in arm phantom experiments. The approach promises improved stability, reproducibility, and hands-free interaction for intraoperative or workflow-constrained vascular ultrasound, with potential for broader ultrasound applications.

Abstract

Medical ultrasound has been widely used to examine vascular structure in modern clinical practice. However, traditional ultrasound examination often faces challenges related to inter- and intra-operator variation. The robotic ultrasound system (RUSS) appears as a potential solution for such challenges because of its superiority in stability and reproducibility. Given the complex anatomy of human vasculature, multiple vessels often appear in ultrasound images, or a single vessel bifurcates into branches, complicating the examination process. To tackle this challenge, this work presents a gaze-guided RUSS for vascular applications. A gaze tracker captures the eye movements of the operator. The extracted gaze signal guides the RUSS to follow the correct vessel when it bifurcates. Additionally, a gaze-guided segmentation network is proposed to enhance segmentation robustness by exploiting gaze information. However, gaze signals are often noisy, requiring interpretation to accurately discern the operator's true intentions. To this end, this study proposes a stabilization module to process raw gaze data. The inferred attention heatmap is utilized as a region proposal to aid segmentation and serve as a trigger signal when the operator needs to adjust the scanning target, such as when a bifurcation appears. To ensure appropriate contact between the probe and surface during scanning, an automatic ultrasound confidence-based orientation correction method is developed. In experiments, we demonstrated the efficiency of the proposed gaze-guided segmentation pipeline by comparing it with other methods. Besides, the performance of the proposed gaze-guided RUSS was also validated as a whole on a realistic arm phantom with an uneven surface.

Figures (5)

• Figure 1: Overview of the Proposed Gaze-Guided Interactive RUSS: The human gaze signal, captured by a gaze tracker, is combined with segmentation history in an intention estimation module to infer the operator's preference, especially when multiple vessels are visible. The resulting attention heatmap guides the gaze-guided segmentation network to produce accurate vessel segmentation masks. These results are integrated into the robotic control loop to keep the target vessel centered in the ultrasound image. A confidence-based orientation correction optimizes probe contact with curved surfaces, improving image quality. Red boxes in ultrasound images highlight shadowed areas caused by improper probe contact.
• Figure 2: (a) The overall design of the proposed gaze-guided segmentation network. (b) The structure of the transformer attention block. (c) The structure of the residual block.
• Figure 3: The design of the human intention estimation module.
• Figure 4: The illustration of confidence-driven probe orientation correction for linear ultrasound probe. The red boxes in the images indicate the shadow areas caused by improper contact. $\{I\}$ represents the ultrasound imaging coordinate system, while $\{P\}$ is the probe coordinate system.
• Figure 5: (a) The experimental setup of the gaze-guided RUSS. (b) Reconstruction results of one representative scanning.