Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The KiTS21 Challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase CT

Nicholas Heller, Fabian Isensee, Dasha Trofimova, Resha Tejpaul, Zhongchen Zhao, Huai Chen, Lisheng Wang, Alex Golts, Daniel Khapun, Daniel Shats, Yoel Shoshan, Flora Gilboa-Solomon, Yasmeen George, Xi Yang, Jianpeng Zhang, Jing Zhang, Yong Xia, Mengran Wu, Zhiyang Liu, Ed Walczak, Sean McSweeney, Ranveer Vasdev, Chris Hornung, Rafat Solaiman, Jamee Schoephoerster, Bailey Abernathy, David Wu, Safa Abdulkadir, Ben Byun, Justice Spriggs, Griffin Struyk, Alexandra Austin, Ben Simpson, Michael Hagstrom, Sierra Virnig, John French, Nitin Venkatesh, Sarah Chan, Keenan Moore, Anna Jacobsen, Susan Austin, Mark Austin, Subodh Regmi, Nikolaos Papanikolopoulos, Christopher Weight

TL;DR

KiTS21 tackles automatic segmentation of kidneys, tumors, and cysts in corticomedullary CT by providing a larger, publicly annotated dataset and an external test set to probe generalization. The challenge combines a transparent multi-annotation workflow, a hierarchical evaluation scheme with Dice and surface Dice, and a peer-reviewed submission requirement to promote reproducibility. Key findings show nnU-Net-based methods dominate top results, with a coarse-to-fine strategy and surface-loss enhancements yielding state-of-the-art performance close to human levels, while hidden-strata analyses reveal performance disparities across subpopulations. The study highlights design choices that improve fairness and interpretability in medical image segmentation challenges and points toward future extensions, including additional imaging phases for KiTS23.

Abstract

This paper presents the challenge report for the 2021 Kidney and Kidney Tumor Segmentation Challenge (KiTS21) held in conjunction with the 2021 international conference on Medical Image Computing and Computer Assisted Interventions (MICCAI). KiTS21 is a sequel to its first edition in 2019, and it features a variety of innovations in how the challenge was designed, in addition to a larger dataset. A novel annotation method was used to collect three separate annotations for each region of interest, and these annotations were performed in a fully transparent setting using a web-based annotation tool. Further, the KiTS21 test set was collected from an outside institution, challenging participants to develop methods that generalize well to new populations. Nonetheless, the top-performing teams achieved a significant improvement over the state of the art set in 2019, and this performance is shown to inch ever closer to human-level performance. An in-depth meta-analysis is presented describing which methods were used and how they faired on the leaderboard, as well as the characteristics of which cases generally saw good performance, and which did not. Overall KiTS21 facilitated a significant advancement in the state of the art in kidney tumor segmentation, and provides useful insights that are applicable to the field of semantic segmentation as a whole.

The KiTS21 Challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase CT

TL;DR

KiTS21 tackles automatic segmentation of kidneys, tumors, and cysts in corticomedullary CT by providing a larger, publicly annotated dataset and an external test set to probe generalization. The challenge combines a transparent multi-annotation workflow, a hierarchical evaluation scheme with Dice and surface Dice, and a peer-reviewed submission requirement to promote reproducibility. Key findings show nnU-Net-based methods dominate top results, with a coarse-to-fine strategy and surface-loss enhancements yielding state-of-the-art performance close to human levels, while hidden-strata analyses reveal performance disparities across subpopulations. The study highlights design choices that improve fairness and interpretability in medical image segmentation challenges and points toward future extensions, including additional imaging phases for KiTS23.

Abstract

This paper presents the challenge report for the 2021 Kidney and Kidney Tumor Segmentation Challenge (KiTS21) held in conjunction with the 2021 international conference on Medical Image Computing and Computer Assisted Interventions (MICCAI). KiTS21 is a sequel to its first edition in 2019, and it features a variety of innovations in how the challenge was designed, in addition to a larger dataset. A novel annotation method was used to collect three separate annotations for each region of interest, and these annotations were performed in a fully transparent setting using a web-based annotation tool. Further, the KiTS21 test set was collected from an outside institution, challenging participants to develop methods that generalize well to new populations. Nonetheless, the top-performing teams achieved a significant improvement over the state of the art set in 2019, and this performance is shown to inch ever closer to human-level performance. An in-depth meta-analysis is presented describing which methods were used and how they faired on the leaderboard, as well as the characteristics of which cases generally saw good performance, and which did not. Overall KiTS21 facilitated a significant advancement in the state of the art in kidney tumor segmentation, and provides useful insights that are applicable to the field of semantic segmentation as a whole.
Paper Structure (44 sections, 1 equation, 12 figures, 4 tables)

This paper contains 44 sections, 1 equation, 12 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Kidney Tumor Background
  3. Kidney Tumor Radiomics
  4. The KiTS21 Challenge
  5. Materials and Methods
  6. The KiTS21 Dataset
  7. Training Set Collection
  8. Test Set Collection
  9. Overall Dataset Characteristics
  10. Data Annotation Process
  11. Public Annotation Platform
  12. Multiple Annotations per Region of Interest
  13. Changes To Intended Segmentation Classes and Dataset Size
  14. Challenge Design Decisions
  15. Use of a Separate Institution for Test Set
  16. ...and 29 more sections

Figures (12)

  • Figure 1: An example axial view of a segmentation showing all three classes represented in this challenge's dataset: 'Kidney' in purple, 'Tumor' in green, and 'Cyst' in blue. Best viewed in color.
  • Figure 2: The geographic locations of the scanning institutions represented in the KiTS21 dataset.
  • Figure 3: A screen capture showing the "Browse" page served to the public on the challenge website. Each case columns for its regions of interest with icons that represent where that case is in the annotation process.
  • Figure 4: A screen capture showing an example of the web-based annotation tool developed for this challenge. When one of the icons shown above is clicked, it brings the user here to view and interact with those annotations.
  • Figure 5: An example of guidance pins (green) placed by a trainee to indicate their annotation intent to guide the laypeople's delineations. Best viewed in color.
  • ...and 7 more figures