Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Intuitive Surgical SurgToolLoc Challenge Results: 2022-2023

Aneeq Zia, Max Berniker, Rogerio Garcia Nespolo, Conor Perreault, Kiran Bhattacharyya, Xi Liu, Ziheng Wang, Satoshi Kondo, Satoshi Kasai, Kousuke Hirasawa, Bo Liu, David Austin, Yiheng Wang, Michal Futrega, Jean-Francois Puget, Zhenqiang Li, Yoichi Sato, Ryo Fujii, Ryo Hachiuma, Mana Masuda, Hideo Saito, An Wang, Mengya Xu, Mobarakol Islam, Long Bai, Winnie Pang, Hongliang Ren, Chinedu Nwoye, Luca Sestini, Nicolas Padoy, Maximilian Nielsen, Samuel Schüttler, Thilo Sentker, Hümeyra Husseini, Ivo Baltruschat, Rüdiger Schmitz, René Werner, Aleksandr Matsun, Mugariya Farooq, Numan Saaed, Jose Renato Restom Viera, Mohammad Yaqub, Neil Getty, Fangfang Xia, Zixuan Zhao, Xiaotian Duan, Xing Yao, Ange Lou, Hao Yang, Jintong Han, Jack Noble, Jie Ying Wu, Tamer Abdulbaki Alshirbaji, Nour Aldeen Jalal, Herag Arabian, Ning Ding, Knut Moeller, Weiliang Chen, Quan He, Muhammad Bilal, Taofeek Akinosho, Adnan Qayyum, Massimo Caputo, Hunaid Vohra, Michael Loizou, Anuoluwapo Ajayi, Ilhem Berrou, Faatihah Niyi-Odumosu, Charlie Budd, Oluwatosin Alabi, Tom Vercauteren, Ruoxi Zhao, Ayberk Acar, John Han, Jumanh Atoum, Yinhong Qin, Jie Ying Wu, Surong Hua, Lu Ping, Wenming Wu, Rongfeng Wei, Jinlin Wu, You Pang, Zhen Chen, Tim Jaspers, Amine Yamlahi, Piotr Kalinowski, Dominik Michael, Tim Rä dsch, Marco Hübner, Danail Stoyanov, Stefanie Speidel, Lena Maier-Hein, Anthony Jarc

TL;DR

This work surveys the SurgToolLoc competitions at MICCAI 2022 and 2023, focusing on weakly supervised surgical tool localization from robotic endoscopic videos using tool-presence labels. It details dataset characteristics, challenge structures, and an array of participant approaches ranging from weakly supervised detection to semi-supervised pipelines that fuse tracking, segmentation, and pseudo-labeling. Across 2022 and 2023, tool presence classification improved more reliably than precise localization, underscoring the difficulty of bounding-box localization under noisy labels and limited frame-level supervision. The study highlights that pretraining on related surgical datasets, along with pseudo-labeling and temporal tracking, are key to performance gains, and it emphasizes the need for scalable self-supervised or semi-supervised strategies to bridge remaining gaps for clinically robust instrument localization. The results establish a foundation for future work in surgical data science, instrument tracking, and AI-assisted robotic surgery planning.

Abstract

Robotic assisted (RA) surgery promises to transform surgical intervention. Intuitive Surgical is committed to fostering these changes and the machine learning models and algorithms that will enable them. With these goals in mind we have invited the surgical data science community to participate in a yearly competition hosted through the Medical Imaging Computing and Computer Assisted Interventions (MICCAI) conference. With varying changes from year to year, we have challenged the community to solve difficult machine learning problems in the context of advanced RA applications. Here we document the results of these challenges, focusing on surgical tool localization (SurgToolLoc). The publicly released dataset that accompanies these challenges is detailed in a separate paper arXiv:2501.09209 [1].

Intuitive Surgical SurgToolLoc Challenge Results: 2022-2023

TL;DR

This work surveys the SurgToolLoc competitions at MICCAI 2022 and 2023, focusing on weakly supervised surgical tool localization from robotic endoscopic videos using tool-presence labels. It details dataset characteristics, challenge structures, and an array of participant approaches ranging from weakly supervised detection to semi-supervised pipelines that fuse tracking, segmentation, and pseudo-labeling. Across 2022 and 2023, tool presence classification improved more reliably than precise localization, underscoring the difficulty of bounding-box localization under noisy labels and limited frame-level supervision. The study highlights that pretraining on related surgical datasets, along with pseudo-labeling and temporal tracking, are key to performance gains, and it emphasizes the need for scalable self-supervised or semi-supervised strategies to bridge remaining gaps for clinically robust instrument localization. The results establish a foundation for future work in surgical data science, instrument tracking, and AI-assisted robotic surgery planning.

Abstract

Robotic assisted (RA) surgery promises to transform surgical intervention. Intuitive Surgical is committed to fostering these changes and the machine learning models and algorithms that will enable them. With these goals in mind we have invited the surgical data science community to participate in a yearly competition hosted through the Medical Imaging Computing and Computer Assisted Interventions (MICCAI) conference. With varying changes from year to year, we have challenged the community to solve difficult machine learning problems in the context of advanced RA applications. Here we document the results of these challenges, focusing on surgical tool localization (SurgToolLoc). The publicly released dataset that accompanies these challenges is detailed in a separate paper arXiv:2501.09209 [1].
Paper Structure (116 sections, 8 equations, 40 figures, 18 tables)

This paper contains 116 sections, 8 equations, 40 figures, 18 tables.

Table of Contents

  1. Introduction
  2. Data Overview
  3. Results and methods from the MICCAI 2022 SurgToolLoc challenge
  4. SurgToolLoc 2022 Challenge Description
  5. Overview
  6. Category 1: Surgical Tool Classification
  7. Category 2: Surgical Tool Classification & Detection
  8. Training Data
  9. Testing Data
  10. Submission Process
  11. Team Submissions
  12. Hikvision Research Institute - Team HRI_MV
  13. Method Description
  14. Preliminary Performance
  15. South China University of China - Team HKMV
  16. ...and 101 more sections

Figures (40)

  • Figure 1: Overview of challenge categories 1 and 2
  • Figure 2: Sample frames with presence labels (left) and a snapshot of the labels CSV file (right)
  • Figure 3: Sample frames with testing labels. The UI interface was blurred to avoid its embedded information (description of the tools) being used for predictions.
  • Figure 4: HRI_MV: The information required for detection and classification is different. Thus, the ROI proposal box output by RPN network was modified.
  • Figure 5: HRI_MV: ROI expand makes model identification device
  • ...and 35 more figures