Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

The Brain Tumor Sequence Registration (BraTS-Reg) Challenge: Establishing Correspondence Between Pre-Operative and Follow-up MRI Scans of Diffuse Glioma Patients

Bhakti Baheti, Satrajit Chakrabarty, Hamed Akbari, Michel Bilello, Benedikt Wiestler, Julian Schwarting, Evan Calabrese, Jeffrey Rudie, Syed Abidi, Mina Mousa, Javier Villanueva-Meyer, Brandon K. K. Fields, Florian Kofler, Russell Takeshi Shinohara, Juan Eugenio Iglesias, Tony C. W. Mok, Albert C. S. Chung, Marek Wodzinski, Artur Jurgas, Niccolo Marini, Manfredo Atzori, Henning Muller, Christoph Grobroehmer, Hanna Siebert, Lasse Hansen, Mattias P. Heinrich, Luca Canalini, Jan Klein, Annika Gerken, Stefan Heldmann, Alessa Hering, Horst K. Hahn, Mingyuan Meng, Lei Bi, Dagan Feng, Jinman Kim, Ramy A. Zeineldin, Mohamed E. Karar, Franziska Mathis-Ullrich, Oliver Burgert, Javid Abderezaei, Aymeric Pionteck, Agamdeep Chopra, Mehmet Kurt, Kewei Yan, Yonghong Yan, Zhe Tang, Jianqiang Ma, Sahar Almahfouz Nasser, Nikhil Cherian Kurian, Mohit Meena, Saqib Shamsi, Amit Sethi, Nicholas J. Tustison, Brian B. Avants, Philip Cook, James C. Gee, Lin Tian, Hastings Greer, Marc Niethammer, Andrew Hoopes, Malte Hoffmann, Adrian V. Dalca, Stergios Christodoulidis, Theo Estiene, Maria Vakalopoulou, Nikos Paragios, Daniel S. Marcus, Christos Davatzikos, Aristeidis Sotiras, Bjoern Menze, Spyridon Bakas, Diana Waldmannstetter

TL;DR

BraTS-Reg establishes the first public benchmark for deformable registration between pre-operative and follow-up brain MRI in diffuse glioma by leveraging landmark-based ground truth and multi-institutional mpMRI data. The challenge combines 70/10/20 train/val/test splits with capsule-like container submissions, evaluating performance via Median Euclidean Error, robustness, and Jacobian-based smoothness, and introduces a BraTS-Reg score that aggregates per-case rankings across metrics. Across ISBI 2022 and MICCAI 2022, top methods converge on a common design: pre-alignment, deep learning with inverse-consistency constraints, and test-time instance optimization, achieving MEE near inter-rater variability for a substantial subset of landmarks but still leaving substantial room for improvement. The results underscore the complexity of tumor-related deformations, highlight the importance of standardized evaluation for cross-method comparisons, and point to future directions including richer annotations and additional qualitative metrics to better capture clinically relevant registration behavior.

Abstract

Registration of longitudinal brain MRI scans containing pathologies is challenging due to dramatic changes in tissue appearance. Although there has been progress in developing general-purpose medical image registration techniques, they have not yet attained the requisite precision and reliability for this task, highlighting its inherent complexity. Here we describe the Brain Tumor Sequence Registration (BraTS-Reg) challenge, as the first public benchmark environment for deformable registration algorithms focusing on estimating correspondences between pre-operative and follow-up scans of the same patient diagnosed with a diffuse brain glioma. The BraTS-Reg data comprise de-identified multi-institutional multi-parametric MRI (mpMRI) scans, curated for size and resolution according to a canonical anatomical template, and divided into training, validation, and testing sets. Clinical experts annotated ground truth (GT) landmark points of anatomical locations distinct across the temporal domain. Quantitative evaluation and ranking were based on the Median Euclidean Error (MEE), Robustness, and the determinant of the Jacobian of the displacement field. The top-ranked methodologies yielded similar performance across all evaluation metrics and shared several methodological commonalities, including pre-alignment, deep neural networks, inverse consistency analysis, and test-time instance optimization per-case basis as a post-processing step. The top-ranked method attained the MEE at or below that of the inter-rater variability for approximately 60% of the evaluated landmarks, underscoring the scope for further accuracy and robustness improvements, especially relative to human experts. The aim of BraTS-Reg is to continue to serve as an active resource for research, with the data and online evaluation tools accessible at https://bratsreg.github.io/.

The Brain Tumor Sequence Registration (BraTS-Reg) Challenge: Establishing Correspondence Between Pre-Operative and Follow-up MRI Scans of Diffuse Glioma Patients

TL;DR

BraTS-Reg establishes the first public benchmark for deformable registration between pre-operative and follow-up brain MRI in diffuse glioma by leveraging landmark-based ground truth and multi-institutional mpMRI data. The challenge combines 70/10/20 train/val/test splits with capsule-like container submissions, evaluating performance via Median Euclidean Error, robustness, and Jacobian-based smoothness, and introduces a BraTS-Reg score that aggregates per-case rankings across metrics. Across ISBI 2022 and MICCAI 2022, top methods converge on a common design: pre-alignment, deep learning with inverse-consistency constraints, and test-time instance optimization, achieving MEE near inter-rater variability for a substantial subset of landmarks but still leaving substantial room for improvement. The results underscore the complexity of tumor-related deformations, highlight the importance of standardized evaluation for cross-method comparisons, and point to future directions including richer annotations and additional qualitative metrics to better capture clinically relevant registration behavior.

Abstract

Registration of longitudinal brain MRI scans containing pathologies is challenging due to dramatic changes in tissue appearance. Although there has been progress in developing general-purpose medical image registration techniques, they have not yet attained the requisite precision and reliability for this task, highlighting its inherent complexity. Here we describe the Brain Tumor Sequence Registration (BraTS-Reg) challenge, as the first public benchmark environment for deformable registration algorithms focusing on estimating correspondences between pre-operative and follow-up scans of the same patient diagnosed with a diffuse brain glioma. The BraTS-Reg data comprise de-identified multi-institutional multi-parametric MRI (mpMRI) scans, curated for size and resolution according to a canonical anatomical template, and divided into training, validation, and testing sets. Clinical experts annotated ground truth (GT) landmark points of anatomical locations distinct across the temporal domain. Quantitative evaluation and ranking were based on the Median Euclidean Error (MEE), Robustness, and the determinant of the Jacobian of the displacement field. The top-ranked methodologies yielded similar performance across all evaluation metrics and shared several methodological commonalities, including pre-alignment, deep neural networks, inverse consistency analysis, and test-time instance optimization per-case basis as a post-processing step. The top-ranked method attained the MEE at or below that of the inter-rater variability for approximately 60% of the evaluated landmarks, underscoring the scope for further accuracy and robustness improvements, especially relative to human experts. The aim of BraTS-Reg is to continue to serve as an active resource for research, with the data and online evaluation tools accessible at https://bratsreg.github.io/.
Paper Structure (40 sections, 9 equations, 9 figures, 4 tables)

This paper contains 40 sections, 9 equations, 9 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Prior Work
  3. Approaches for Baseline-to-follow-up Registration
  4. Related Challenges and Benchmarks
  5. Existing Datasets for Deformable Registration and Baseline-to-follow-up Registration
  6. Challenge Description
  7. Overview
  8. Multi-institutional Data Sources
  9. Data Preparation
  10. Landmark Annotation Protocol
  11. Inter-Rater Annotation Variability
  12. Data Partitioning in Distinct Independent Cohorts
  13. Training Data
  14. Validation Data
  15. Test Data
  16. ...and 25 more sections

Figures (9)

  • Figure 1: Example of a pre-operative baseline and its corresponding follow-up MRI scan. The contrast-enhanced T1-weighted (T1-CE), and the T2 Fluid Attenuated Inversion Recovery (FLAIR) baseline scan clearly show the tumor and the edema, respectively. Similarly, the T1-CE follow-up scan shows the resection cavity, whereas the edema is visible in the FLAIR scan.
  • Figure 2: Distribution of time gap between baseline and followup scan over the complete data used in BraTS-Reg challenge. Each bin of the histogram indicates one month starting from 0 to 48.
  • Figure 3: An example of corresponding pair of landmark point in Baseline (left) and Follow-up (Right) superimposed on t1ce scan for visualization
  • Figure 4: Inter-rater analysis. (a) shows the distribution of inter-rater annotation variability over a selection of test cases, with $median=1.41$ and $mean=1.77$. This variability is correlated to the respective landmark-tumor distance (blue) in (b). The respective regression line (red) and a Pearson correlation coefficient of $r=-0.13$ show a small correlation between annotation variability and landmark-tumor distance.
  • Figure 5: Comparative performance analysis of various participating methods in terms of Median Euclidean Error (MEE), Robustness and BraTS-Reg score along with the invited methods (indicated with *). The white line in the violin plots indicates the median, whereas the red cross indicates the mean of the distribution.
  • ...and 4 more figures