Surgical Tattoos in Infrared: A Dataset for Quantifying Tissue Tracking and Mapping

TL;DR

A novel labeling methodology along with a dataset that uses said methodology, Surgical Tattoos in Infrared (STIR), has labels that are persistent but invisible to visible spectrum algorithms, which will help to quantify and enable better analysis of tracking and mapping methods.

Abstract

Quantifying performance of methods for tracking and mapping tissue in endoscopic environments is essential for enabling image guidance and automation of medical interventions and surgery. Datasets developed so far either use rigid environments, visible markers, or require annotators to label salient points in videos after collection. These are respectively: not general, visible to algorithms, or costly and error-prone. We introduce a novel labeling methodology along with a dataset that uses said methodology, Surgical Tattoos in Infrared (STIR). STIR has labels that are persistent but invisible to visible spectrum algorithms. This is done by labelling tissue points with IR-fluorescent dye, indocyanine green (ICG), and then collecting visible light video clips. STIR comprises hundreds of stereo video clips in both in-vivo and ex-vivo scenes with start and end points labelled in the IR spectrum. With over 3,000 labelled points, STIR will help to quantify and enable better analysis of tracking and mapping methods. After introducing STIR, we analyze multiple different frame-based tracking methods on STIR using both 3D and 2D endpoint error and accuracy metrics. STIR is available at https://dx.doi.org/10.21227/w8g4-g548

Figures (18)

• Figure 1: Labelling process: Demonstration of what the labelling process for each clip looks like. Data Format: An example sequence from the released STIR dataset. Each sequence includes two visible spectrum video clips for the left and right stereo views, IR images at the start and end of the clip, and labels of the start and end segments in the left frame. Specific segments can be seen in the circular cutouts. Right: A set of random start and end IR ground truth frame sample pairs from STIR. The center points of each segment are marked with a red circle.
• Figure 2: Left to right: sterile water, ICG, IR ink, and IR pen used for labelling tissue.
• Figure 3: Different means to apply ink to tissue surface. Top: Microneedling device. Middle: Close-ups of needle array on microneedle grid. Bottom: Hypodermic needle with a tattoo needle inserted into the hollow.
• Figure 4: Types of labelling. The tattoo needle, IR-ink marker, and bead insertion method are shown. Two captures of visible/IR pairs are shown for each. Note how the bump of the beads or the ink the marker pen are both visible in the visible spectrum images.
• Figure 5: India ink microneedle array tattooing results on a chicken breast. On the left are failed dye applications due to uneven ink. On the right are successful dye applications.