Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Amodal Segmentation for Laparoscopic Surgery Video Instruments

Ruohua Shi, Zhaochen Liu, Lingyu Duan, Tingting Jiang

TL;DR

This study addresses occlusion in laparoscopic instrument segmentation by applying amodal segmentation to predict both visible and occluded parts of surgical tools. It introduces the AIS dataset, derived from the 2017 EndoVis Robotic Instrument Segmentation Challenge, with full amodal masks for 7,084 instrument instances at 1024×1280, and benchmarks four amodal methods (SAM, AISFormer, C2F-Seg, PLUG) on this medical domain. PLUG achieves the top mean IoU of 89.25, demonstrating the feasibility and value of amodal segmentation for surgical workflows. The work enables improved intraoperative guidance, post hoc procedure analysis, and education by revealing complete instrument shapes and trajectories even under occlusion.

Abstract

Segmentation of surgical instruments is crucial for enhancing surgeon performance and ensuring patient safety. Conventional techniques such as binary, semantic, and instance segmentation share a common drawback: they do not accommodate the parts of instruments obscured by tissues or other instruments. Precisely predicting the full extent of these occluded instruments can significantly improve laparoscopic surgeries by providing critical guidance during operations and assisting in the analysis of potential surgical errors, as well as serving educational purposes. In this paper, we introduce Amodal Segmentation to the realm of surgical instruments in the medical field. This technique identifies both the visible and occluded parts of an object. To achieve this, we introduce a new Amoal Instruments Segmentation (AIS) dataset, which was developed by reannotating each instrument with its complete mask, utilizing the 2017 MICCAI EndoVis Robotic Instrument Segmentation Challenge dataset. Additionally, we evaluate several leading amodal segmentation methods to establish a benchmark for this new dataset.

Amodal Segmentation for Laparoscopic Surgery Video Instruments

TL;DR

This study addresses occlusion in laparoscopic instrument segmentation by applying amodal segmentation to predict both visible and occluded parts of surgical tools. It introduces the AIS dataset, derived from the 2017 EndoVis Robotic Instrument Segmentation Challenge, with full amodal masks for 7,084 instrument instances at 1024×1280, and benchmarks four amodal methods (SAM, AISFormer, C2F-Seg, PLUG) on this medical domain. PLUG achieves the top mean IoU of 89.25, demonstrating the feasibility and value of amodal segmentation for surgical workflows. The work enables improved intraoperative guidance, post hoc procedure analysis, and education by revealing complete instrument shapes and trajectories even under occlusion.

Abstract

Segmentation of surgical instruments is crucial for enhancing surgeon performance and ensuring patient safety. Conventional techniques such as binary, semantic, and instance segmentation share a common drawback: they do not accommodate the parts of instruments obscured by tissues or other instruments. Precisely predicting the full extent of these occluded instruments can significantly improve laparoscopic surgeries by providing critical guidance during operations and assisting in the analysis of potential surgical errors, as well as serving educational purposes. In this paper, we introduce Amodal Segmentation to the realm of surgical instruments in the medical field. This technique identifies both the visible and occluded parts of an object. To achieve this, we introduce a new Amoal Instruments Segmentation (AIS) dataset, which was developed by reannotating each instrument with its complete mask, utilizing the 2017 MICCAI EndoVis Robotic Instrument Segmentation Challenge dataset. Additionally, we evaluate several leading amodal segmentation methods to establish a benchmark for this new dataset.
Paper Structure (14 sections, 5 figures, 2 tables)

This paper contains 14 sections, 5 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Surgery Video Instrument Segmentation
  4. Amodal Segmentation
  5. Dataset
  6. Dataset Acquisition
  7. Dataset Analysis
  8. Benchmarks
  9. Selected Methods
  10. Evaluation Metrics
  11. Experimental Setup
  12. Results
  13. Performance comparison
  14. Hard Case

Figures (5)

  • Figure 1: Segmentation examples. (1) Frames from laparoscopic surgery; (2) Instance segmentation ground truth; (3) Segmentation masks with occluded parts.
  • Figure 2: Illustration of annotation process. (a) displays a polygon annotation formed by various key points and line segments. Once the labeling of this frame is complete, the annotation can be carried forward to the next frame for further labeling, as depicted in (b). By adjusting the positions of the key points within the polygon, the final annotation of the instrument in (b) is presented in (c).
  • Figure 3: The input mode of bounding boxes. In order to reduce the number of manual inputs, we use the bounding box of the predicted mask in the previous frame as the input bounding box of the next frame when possible.
  • Figure 4: An example of the excessively occluded case. The instrument is almost completely occluded in this frame.
  • Figure 5: Qualitative results. The qualitative comparison of predicted amodal masks from SAM, AISFormer, C2F-Seg and PLUG. These ten rows are chosen from the first to tenth sub-testset in order from top to bottom.