Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Novel Tracking Framework for Devices in X-ray Leveraging Supplementary Cue-Driven Self-Supervised Features

Saahil Islam, Venkatesh N. Murthy, Dominik Neumann, Serkan Cimen, Puneet Sharma, Andreas Maier, Dorin Comaniciu, Florin C. Ghesu

TL;DR

This work tackles robust, real-time tracking of small interventional X-ray devices such as balloon markers and catheter tips under challenging occlusions. It introduces a self-supervised spatio-temporal learning framework that injects supplementary vesselness cues via a weak-label objective (FIMAE-SC) trained on a large unlabeled dataset, enabling richer representations. A Historical Feature Guided Tracker (HiFT) then leverages these pretrained features with symmetric crops and historical trajectories through cross-attention to localize multiple landmarks accurately. The approach achieves state-of-the-art robustness, with major reductions in max error for balloon markers and catheter tips, and shows robustness under occlusion, supported by ablations that validate the roles of appearance and trajectory information and pretraining.

Abstract

To restore proper blood flow in blocked coronary arteries via angioplasty procedure, accurate placement of devices such as catheters, balloons, and stents under live fluoroscopy or diagnostic angiography is crucial. Identified balloon markers help in enhancing stent visibility in X-ray sequences, while the catheter tip aids in precise navigation and co-registering vessel structures, reducing the need for contrast in angiography. However, accurate detection of these devices in interventional X-ray sequences faces significant challenges, particularly due to occlusions from contrasted vessels and other devices and distractions from surrounding, resulting in the failure to track such small objects. While most tracking methods rely on spatial correlation of past and current appearance, they often lack strong motion comprehension essential for navigating through these challenging conditions, and fail to effectively detect multiple instances in the scene. To overcome these limitations, we propose a self-supervised learning approach that enhances its spatio-temporal understanding by incorporating supplementary cues and learning across multiple representation spaces on a large dataset. Followed by that, we introduce a generic real-time tracking framework that effectively leverages the pretrained spatio-temporal network and also takes the historical appearance and trajectory data into account. This results in enhanced localization of multiple instances of device landmarks. Our method outperforms state-of-the-art methods in interventional X-ray device tracking, especially stability and robustness, achieving an 87% reduction in max error for balloon marker detection and a 61% reduction in max error for catheter tip detection.

A Novel Tracking Framework for Devices in X-ray Leveraging Supplementary Cue-Driven Self-Supervised Features

TL;DR

This work tackles robust, real-time tracking of small interventional X-ray devices such as balloon markers and catheter tips under challenging occlusions. It introduces a self-supervised spatio-temporal learning framework that injects supplementary vesselness cues via a weak-label objective (FIMAE-SC) trained on a large unlabeled dataset, enabling richer representations. A Historical Feature Guided Tracker (HiFT) then leverages these pretrained features with symmetric crops and historical trajectories through cross-attention to localize multiple landmarks accurately. The approach achieves state-of-the-art robustness, with major reductions in max error for balloon markers and catheter tips, and shows robustness under occlusion, supported by ablations that validate the roles of appearance and trajectory information and pretraining.

Abstract

To restore proper blood flow in blocked coronary arteries via angioplasty procedure, accurate placement of devices such as catheters, balloons, and stents under live fluoroscopy or diagnostic angiography is crucial. Identified balloon markers help in enhancing stent visibility in X-ray sequences, while the catheter tip aids in precise navigation and co-registering vessel structures, reducing the need for contrast in angiography. However, accurate detection of these devices in interventional X-ray sequences faces significant challenges, particularly due to occlusions from contrasted vessels and other devices and distractions from surrounding, resulting in the failure to track such small objects. While most tracking methods rely on spatial correlation of past and current appearance, they often lack strong motion comprehension essential for navigating through these challenging conditions, and fail to effectively detect multiple instances in the scene. To overcome these limitations, we propose a self-supervised learning approach that enhances its spatio-temporal understanding by incorporating supplementary cues and learning across multiple representation spaces on a large dataset. Followed by that, we introduce a generic real-time tracking framework that effectively leverages the pretrained spatio-temporal network and also takes the historical appearance and trajectory data into account. This results in enhanced localization of multiple instances of device landmarks. Our method outperforms state-of-the-art methods in interventional X-ray device tracking, especially stability and robustness, achieving an 87% reduction in max error for balloon marker detection and a 61% reduction in max error for catheter tip detection.
Paper Structure (7 sections, 5 equations, 4 figures, 3 tables)

This paper contains 7 sections, 5 equations, 4 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Methods
  3. Self-Supervised Learning with Supplementary Cues
  4. Historical Feature Guided Tracker
  5. Experiments
  6. Conclusion
  7. Disclosure of Interests.

Figures (4)

  • Figure 1: Example of (a) balloon markers and (b) catheter tip highlighted in black: indicating the change in appearance over time with the contrast flowing through the vessels.
  • Figure 2: Overview of our framework: (a) The Self-Supervised Learning (pretraining) and (b) Historical Guided Tracker leveraging the pretrained features.
  • Figure 3: Error distribution for scenarios with and without occlusion for (a) catheter tip and (b) balloon marker tracking
  • Figure 4: Qualitative examples of balloon marker and catheter tip tracking: Robust performance of HiFT across occlusions and distractors compared to SimST.