Needle Segmentation Using GAN: Restoring Thin Instrument Visibility in Robotic Ultrasound

TL;DR

This work tackles the critical problem of maintaining needle visibility during ultrasound-guided percutaneous procedures by introducing AdvSeg-Net, a GAN-based needle segmentation framework that enforces high-order consistency between predicted masks and ground truth. The system includes a two-phase training strategy, a slice-thickness correction method for transverse scanning, and an autonomous robotic workflow that detects misalignment and repositions the US probe to restore visibility. Validated on ex vivo porcine tissue, the approach achieves precise needle localization with low tip and angle errors and successfully restores visibility in all trials, demonstrating the practicality of real-time, autonomous image-guided robotic intervention. The combination of a UNet++ generator, Dice+Contextual Loss, and adversarial refinement yields robust segmentation while enabling real-time performance for clinical workflows.

Abstract

Ultrasound-guided percutaneous needle insertion is a standard procedure employed in both biopsy and ablation in clinical practices. However, due to the complex interaction between tissue and instrument, the needle may deviate from the in-plane view, resulting in a lack of close monitoring of the percutaneous needle. To address this challenge, we introduce a robot-assisted ultrasound (US) imaging system designed to seamlessly monitor the insertion process and autonomously restore the visibility of the inserted instrument when misalignment happens. To this end, the adversarial structure is presented to encourage the generation of segmentation masks that align consistently with the ground truth in high-order space. This study also systematically investigates the effects on segmentation performance by exploring various training loss functions and their combinations. When misalignment between the probe and the percutaneous needle is detected, the robot is triggered to perform transverse searching to optimize the positional and rotational adjustment to restore needle visibility. The experimental results on ex-vivo porcine samples demonstrate that the proposed method can precisely segment the percutaneous needle (with a tip error of $0.37\pm0.29mm$ and an angle error of $1.19\pm 0.29^{\circ}$). Furthermore, the needle appearance can be successfully restored under the repositioned probe pose in all 45 trials, with repositioning errors of $1.51\pm0.95mm$ and $1.25\pm0.79^{\circ}$. from latex to text with math symbols

Figures (7)

• Figure 1: Illustration of needle insertion on an ex vivo sample of porcine tissue. (a) is the overall scene. (b) and (e) refer to the misalignment case and the desired case, respectively. (c) and (d) are the corresponding B-mode images obtained in the case of (b) and (e), respectively.
• Figure 2: (a) Illustration of the proposed AdvSeg-Net for thin instruments segmentation in US images. (b) Detailed structure for the discriminator.
• Figure 3: The schematic illustration of the probe repositioning.
• Figure 4: Illustration of US slice thickness artifact with respect to varying $\theta_{mis}$. This representative result is obtained using a water tank. (a)-(e) The real setup and corresponding US images when $\theta_{mis}$ is $0$, $10^{\circ}$, $20^{\circ}$, $40^{\circ}$, and $90^{\circ}$, respectively. (f) The computed SSIM between the last frame and all other frames. The orange rectangle is a pre-defined region of interest to limit the negative impact caused by irrelevant US artifacts.
• Figure 5: (a) Illustration of the real scene. (b) Top-down view along the Z axis of probe frame $\{p\}$. The solid and dashed rectangles represented the original and adjusted probe pose, respectively.