Analysis of the BraTS 2023 Intracranial Meningioma Segmentation Challenge

TL;DR

BraTS 2023 extends the BraTS framework to intracranial meningioma segmentation, leveraging a large multi-institution, expert-annotated dataset with multi-sequence MRI (T1, T2, FLAIR, T1Gd) and lesion-wise evaluation using $DSC$ and $HD_{95}$. Nine teams developed deep learning segmentation models, achieving state-of-the-art lesion-wise performance with top results around $DSC$ values near 0.98 for individual lesions and average DSCs around 0.87–0.90 across ET, TC, and WT, while revealing preprocessing vulnerabilities due to skull-stripping that abuts tumor voxels in the majority of cases. The evaluation framework, including lesion-wise scoring, 26-connectivity lesion separation, and federated MedPerf-based testing, provides granular insight into per-lesion performance and robustness across tumor sub-regions. Key caveats include poor performance on heavily calcified non-enhancing lesions and the need for external validation and multimodal imaging to improve generalization and clinical applicability. Overall, the work sets a rigorous benchmark for pre-operative meningioma segmentation and outlines clear directions for advancing preprocessing, multimodal data integration, and cross-institutional reliability.

Abstract

We describe the design and results from the BraTS 2023 Intracranial Meningioma Segmentation Challenge. The BraTS Meningioma Challenge differed from prior BraTS Glioma challenges in that it focused on meningiomas, which are typically benign extra-axial tumors with diverse radiologic and anatomical presentation and a propensity for multiplicity. Nine participating teams each developed deep-learning automated segmentation models using image data from the largest multi-institutional systematically expert annotated multilabel multi-sequence meningioma MRI dataset to date, which included 1000 training set cases, 141 validation set cases, and 283 hidden test set cases. Each case included T2, FLAIR, T1, and T1Gd brain MRI sequences with associated tumor compartment labels delineating enhancing tumor, non-enhancing tumor, and surrounding non-enhancing FLAIR hyperintensity. Participant automated segmentation models were evaluated and ranked based on a scoring system evaluating lesion-wise metrics including dice similarity coefficient (DSC) and 95% Hausdorff Distance. The top ranked team had a lesion-wise median dice similarity coefficient (DSC) of 0.976, 0.976, and 0.964 for enhancing tumor, tumor core, and whole tumor, respectively and a corresponding average DSC of 0.899, 0.904, and 0.871, respectively. These results serve as state-of-the-art benchmarks for future pre-operative meningioma automated segmentation algorithms. Additionally, we found that 1286 of 1424 cases (90.3%) had at least 1 compartment voxel abutting the edge of the skull-stripped image edge, which requires further investigation into optimal pre-processing face anonymization steps.

Figures (12)

• Figure 1: Axial (left) and coronal (right) views of meningiomas at the most common locations in the skull. This is a modified figure as adapted from Murek under the CC-BY-4.0 license
• Figure 2: Meningioma sub-compartments considered in the BraTS Pre-operative Meningioma Dataset. Image panels A-C denote the different tumor sub-compartments included in manual annotations; (A) enhancing tumor (blue) visible on a T1-weighted post-contrast image; (B) the non-enhancing tumor core (red) visible on a T1-weighted post-contrast image; (C) the surrounding FLAIR hyperintensity (green) visible on a FLAIR image; (D) combined segmentations generating the final tumor sub-compartment labels provided in the BraTS Pre-operative Meningioma Dataset.
• Figure 3: Violin plots of DSC and 95HD scores for the ET, TC, and WT regions across all of the participating teams. The subplots are organized as: A1 (ET DSC), A2 (TC DSC), A3 (WT DSC), B1 (ET 95HD), B2 (95HD TC), B3 (95HD WT).
• Figure 4: Image panels demonstrating the different predictions of the top 3 teams for a pre-operative meningioma testing set case as seen on T1Gd (top row) and FLAIR (bottom row) MRI.
• Figure 5: Image panels of the top scored individual test case with a median participant DSC of 1.00, 1.00, and 1.00 and average participant DSC of 0.993, 0.881, and 0.882 for enhancing tumor, tumor core, and whole tumor; respectively. Tumor ground truth sub-compartment labels annotated on axial (A), sagittal (B), and coronal (C) views of a T1Gd MRI head case. Panel D depicts the tumor abutting the edge of the skull-stripped brain without a label.