Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

2018 Robotic Scene Segmentation Challenge

Max Allan, Satoshi Kondo, Sebastian Bodenstedt, Stefan Leger, Rahim Kadkhodamohammadi, Imanol Luengo, Felix Fuentes, Evangello Flouty, Ahmed Mohammed, Marius Pedersen, Avinash Kori, Varghese Alex, Ganapathy Krishnamurthi, David Rauber, Robert Mendel, Christoph Palm, Sophia Bano, Guinther Saibro, Chi-Sheng Shih, Hsun-An Chiang, Juntang Zhuang, Junlin Yang, Vladimir Iglovikov, Anton Dobrenkii, Madhu Reddiboina, Anubhav Reddy, Xingtong Liu, Cong Gao, Mathias Unberath, Myeonghyeon Kim, Chanho Kim, Chaewon Kim, Hyejin Kim, Gyeongmin Lee, Ihsan Ullah, Miguel Luna, Sang Hyun Park, Mahdi Azizian, Danail Stoyanov, Lena Maier-Hein, Stefanie Speidel

TL;DR

This paper presents the 2018 EndoVis robotic scene segmentation challenge, expanding beyond prior instrument-focused tasks to include anatomical and medical-device classes using porcine laparoscopic data. It catalogs a diverse set of segmentation approaches (DeepLabV3+, PSPNet, U-Nets, and stereo-aware architectures) and a broad array of training strategies across 19 sequences (15 train, 4 test). The study demonstrates mixed success across classes, with kidney-related structures and heavily occluded surfaces posing the greatest challenges, and identifies Regensburg as a leading performer across multiple datasets. Overall, the work advances surgical scene understanding by benchmarking multi-class, multi-object segmentation in realistic MIS imagery and highlighting avenues for robustness under occlusion, class imbalance, and domain shift, enabling improved AR overlays and decision support in robotic surgery.

Abstract

In 2015 we began a sub-challenge at the EndoVis workshop at MICCAI in Munich using endoscope images of ex-vivo tissue with automatically generated annotations from robot forward kinematics and instrument CAD models. However, the limited background variation and simple motion rendered the dataset uninformative in learning about which techniques would be suitable for segmentation in real surgery. In 2017, at the same workshop in Quebec we introduced the robotic instrument segmentation dataset with 10 teams participating in the challenge to perform binary, articulating parts and type segmentation of da Vinci instruments. This challenge included realistic instrument motion and more complex porcine tissue as background and was widely addressed with modifications on U-Nets and other popular CNN architectures. In 2018 we added to the complexity by introducing a set of anatomical objects and medical devices to the segmented classes. To avoid over-complicating the challenge, we continued with porcine data which is dramatically simpler than human tissue due to the lack of fatty tissue occluding many organs.

2018 Robotic Scene Segmentation Challenge

TL;DR

This paper presents the 2018 EndoVis robotic scene segmentation challenge, expanding beyond prior instrument-focused tasks to include anatomical and medical-device classes using porcine laparoscopic data. It catalogs a diverse set of segmentation approaches (DeepLabV3+, PSPNet, U-Nets, and stereo-aware architectures) and a broad array of training strategies across 19 sequences (15 train, 4 test). The study demonstrates mixed success across classes, with kidney-related structures and heavily occluded surfaces posing the greatest challenges, and identifies Regensburg as a leading performer across multiple datasets. Overall, the work advances surgical scene understanding by benchmarking multi-class, multi-object segmentation in realistic MIS imagery and highlighting avenues for robustness under occlusion, class imbalance, and domain shift, enabling improved AR overlays and decision support in robotic surgery.

Abstract

In 2015 we began a sub-challenge at the EndoVis workshop at MICCAI in Munich using endoscope images of ex-vivo tissue with automatically generated annotations from robot forward kinematics and instrument CAD models. However, the limited background variation and simple motion rendered the dataset uninformative in learning about which techniques would be suitable for segmentation in real surgery. In 2017, at the same workshop in Quebec we introduced the robotic instrument segmentation dataset with 10 teams participating in the challenge to perform binary, articulating parts and type segmentation of da Vinci instruments. This challenge included realistic instrument motion and more complex porcine tissue as background and was widely addressed with modifications on U-Nets and other popular CNN architectures. In 2018 we added to the complexity by introducing a set of anatomical objects and medical devices to the segmented classes. To avoid over-complicating the challenge, we continued with porcine data which is dramatically simpler than human tissue due to the lack of fatty tissue occluding many organs.

Paper Structure

This paper contains 28 sections, 2 equations, 9 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Data
  3. Challenge Overview
  4. Data Collection
  5. Data Annotation
  6. Participating Methods
  7. Konika Minola
  8. National Center for Tumor Diseases
  9. Digital Surgery
  10. Norwegian University of Science and Technology
  11. Indian Institute of Technology, Madras
  12. Ostbayerische Technische Hochschule Regensburg
  13. University College London (UCL)
  14. National Center for Tumor Diseases (NCT)
  15. IRCAD
  16. ...and 13 more sections

Figures (9)

  • Figure 1: An example of the fusion of segmented blood vessels from pre-operative CT imaging with the endoscopic view.
  • Figure 2: The different classes to be segmented in our challenge. The instrument and ultrasound probes were common classes with our 2017 dataset. All other classes were new to this challenge.
  • Figure 3: Example frames from the training datasets in order from left to right: Dataset 1, 4, 5, 7, 9, 12, 13, 14.
  • Figure 4: As the fascia is stretched off the kidney, the case of exactly where the border of the covered kidney label should now lie is fairly complex to describe in a consistent way.
  • Figure 5: Example annotations from the training datasets in order from left to right: Dataset 1, 4, 5, 7, 9, 12, 13, 14.
  • ...and 4 more figures