Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformers

Alex Ranne, Liming Kuang, Yordanka Velikova, Nassir Navab, Ferdinando Rodriguez y Baena

TL;DR

A self-supervised deep learning architecture is introduced to segment catheters in longitudinal ultrasound images, without demanding any labeled data, to address both problems by introducing a self-supervised deep learning architecture to segment catheters in longitudinal ultrasound images.

Abstract

In minimally invasive endovascular procedures, contrast-enhanced angiography remains the most robust imaging technique. However, it is at the expense of the patient and clinician's health due to prolonged radiation exposure. As an alternative, interventional ultrasound has notable benefits such as being radiation-free, fast to deploy, and having a small footprint in the operating room. Yet, ultrasound is hard to interpret, and highly prone to artifacts and noise. Additionally, interventional radiologists must undergo extensive training before they become qualified to diagnose and treat patients effectively, leading to a shortage of staff, and a lack of open-source datasets. In this work, we seek to address both problems by introducing a self-supervised deep learning architecture to segment catheters in longitudinal ultrasound images, without demanding any labeled data. The network architecture builds upon AiAReSeg, a segmentation transformer built with the Attention in Attention mechanism, and is capable of learning feature changes across time and space. To facilitate training, we used synthetic ultrasound data based on physics-driven catheter insertion simulations, and translated the data into a unique CT-Ultrasound common domain, CACTUSS, to improve the segmentation performance. We generated ground truth segmentation masks by computing the optical flow between adjacent frames using FlowNet2, and performed thresholding to obtain a binary map estimate. Finally, we validated our model on a test dataset, consisting of unseen synthetic data and images collected from silicon aorta phantoms, thus demonstrating its potential for applications to clinical data in the future.

CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformers

TL;DR

A self-supervised deep learning architecture is introduced to segment catheters in longitudinal ultrasound images, without demanding any labeled data, to address both problems by introducing a self-supervised deep learning architecture to segment catheters in longitudinal ultrasound images.

Abstract

In minimally invasive endovascular procedures, contrast-enhanced angiography remains the most robust imaging technique. However, it is at the expense of the patient and clinician's health due to prolonged radiation exposure. As an alternative, interventional ultrasound has notable benefits such as being radiation-free, fast to deploy, and having a small footprint in the operating room. Yet, ultrasound is hard to interpret, and highly prone to artifacts and noise. Additionally, interventional radiologists must undergo extensive training before they become qualified to diagnose and treat patients effectively, leading to a shortage of staff, and a lack of open-source datasets. In this work, we seek to address both problems by introducing a self-supervised deep learning architecture to segment catheters in longitudinal ultrasound images, without demanding any labeled data. The network architecture builds upon AiAReSeg, a segmentation transformer built with the Attention in Attention mechanism, and is capable of learning feature changes across time and space. To facilitate training, we used synthetic ultrasound data based on physics-driven catheter insertion simulations, and translated the data into a unique CT-Ultrasound common domain, CACTUSS, to improve the segmentation performance. We generated ground truth segmentation masks by computing the optical flow between adjacent frames using FlowNet2, and performed thresholding to obtain a binary map estimate. Finally, we validated our model on a test dataset, consisting of unseen synthetic data and images collected from silicon aorta phantoms, thus demonstrating its potential for applications to clinical data in the future.
Paper Structure (25 sections, 2 equations, 5 figures, 2 tables)

This paper contains 25 sections, 2 equations, 5 figures, 2 tables.

Table of Contents

  1. INTRODUCTION
  2. RELATED WORKS
  3. State of the Art in Image Segmentation
  4. Deep Learning: CNN-Based Methods
  5. Deep Learning: Transformer and Hybrid Methods
  6. Catheter detection and tracking
  7. AiAReSeg and the Attention in Attention Mechanism
  8. Catheter Simulations and CACTUSS
  9. Unsupervised motion segmentation
  10. METHODOLOGY
  11. Data Acquisition
  12. CathFlow: Self-supervised segmentation
  13. Data Processing
  14. Inference
  15. EXPERIMENTAL EVALUATIONS
  16. ...and 10 more sections

Figures (5)

  • Figure 1: Proposed Interventional Workflow when CathFlow is incorporated into the surgical workflow
  • Figure 2: Detailed pipeline of CathFlow. Representative inputs and outputs to each module are shown in the diagram. The K, V and Q, explained in ranne2023aiareseg, correspond to the keys, values and queries, respectively. K are feature maps from the initial and intermediate frames, V are the K masked using segmentation results from the previous frame, and Q are features from the current search frame.
  • Figure 3: Phantom setup for collecting images
  • Figure 4: The Inference Pipeline of CathFlow
  • Figure 5: Different optical flow generation methods and respective generated optical flow on ultrasound sequence.