EgoSurgery-Tool: A Dataset of Surgical Tool and Hand Detection from Egocentric Open Surgery Videos

TL;DR

EgoSurgery-Tool is introduced, an extension of the existing EgoSurgery-Phase dataset, which contains real open surgery videos captured using an egocentric camera attached to the surgeon's head, along with phase annotations, and is superior to existing datasets due to its larger scale, greater variety of surgical tools, more annotations, and denser scenes.

Abstract

Surgical tool detection is a fundamental task for understanding egocentric open surgery videos. However, detecting surgical tools presents significant challenges due to their highly imbalanced class distribution, similar shapes and similar textures, and heavy occlusion. The lack of a comprehensive large-scale dataset compounds these challenges. In this paper, we introduce EgoSurgery-Tool, an extension of the existing EgoSurgery-Phase dataset, which contains real open surgery videos captured using an egocentric camera attached to the surgeon's head, along with phase annotations. EgoSurgery-Tool has been densely annotated with surgical tools and comprises over 49K surgical tool bounding boxes across 15 categories, constituting a large-scale surgical tool detection dataset. EgoSurgery-Tool also provides annotations for hand detection with over 46K hand-bounding boxes, capturing hand-object interactions that are crucial for understanding activities in egocentric open surgery. EgoSurgery-Tool is superior to existing datasets due to its larger scale, greater variety of surgical tools, more annotations, and denser scenes. We conduct a comprehensive analysis of EgoSurgery-Tool using nine popular object detectors to assess their effectiveness in both surgical tool and hand detection. The dataset will be released at https://github.com/Fujiry0/EgoSurgery.

Figures (6)

• Figure 1: Visualizations of EgoSurgery-Tool and ground truth annotations, for surgical tools, own left hand, own right hand, other left hands, and other right hands.
• Figure 2: The distribution of surgical tool categories.
• Figure 3: The distribution of hand categories.
• Figure 4: Co-occurrence matrix between surgical tools and surgical phases.
• Figure 5: Qualitative results for the object detection challenge. The first column shows correct detections, while the second column shows incorrect cases.