The Brain Tumor Segmentation (BraTS) Challenge: Local Synthesis of Healthy Brain Tissue via Inpainting

TL;DR

BraTS inpainting extends BraTS by tackling local synthesis of healthy brain tissue within tumor regions on 3D T1 MRIs, enabling standard brain analysis tools to operate on lesioned scans. The paper defines data generation, mask creation, manual curation, and evaluation procedures, and reports on the 2023 challenge including participant methods and robustness analyses. It notes the addition of a meningioma test set in 2024 to probe generalization, and discusses barriers to participation, the need for better synthesis metrics, and the potential of 3D diffusion models for improved inpaintings. Overall, the work establishes a benchmark for 3D MRI inpainting in the neuroimaging domain and provides insights into effective evaluation strategies and future directions.

Abstract

A myriad of algorithms for the automatic analysis of brain MR images is available to support clinicians in their decision-making. For brain tumor patients, the image acquisition time series typically starts with an already pathological scan. This poses problems, as many algorithms are designed to analyze healthy brains and provide no guarantee for images featuring lesions. Examples include, but are not limited to, algorithms for brain anatomy parcellation, tissue segmentation, and brain extraction. To solve this dilemma, we introduce the BraTS inpainting challenge. Here, the participants explore inpainting techniques to synthesize healthy brain scans from lesioned ones. The following manuscript contains the task formulation, dataset, and submission procedure. Later, it will be updated to summarize the findings of the challenge. The challenge is organized as part of the ASNR-BraTS MICCAI challenge.

Figures (8)

• Figure 1: Challenge Task: Given a set of pathological and incomplete T1 MRI images with specified (masked) voided areas, the participants are tasked to synthesize healthy versions of the pathological brains. We cast the problem of synthesizing the healthy tissue as inpainting within the tumor area.
• Figure 2: Dataset figure: Here, we depict the properties of the available imaging modalities at each stage of the challenge. During all phases, we provide a T1 image with voided regions and a corresponding void mask. Only, during training we provide the corresponding whole T1 image with an additional mask for the tumor and healthy area. These masks are provided to help participants with sampling healthy tissue. However, we explicitly encourage participants to experiment with better sampling strategies.
• Figure 3: Healthy inpainting masks in the RSNA-ASNR-MICCAI BraTS 2023 inpainting challenge: Exemplary healthy mask generation procedure for one brain and one healthy mask. Panel A shows the underlying T1 scan and panel B has the respective tumor annotation overlaid in red). T1 scan and tumor annotation are the input for our mask sampling algorithm (right side. The algorithm randomly samples a mask, transforms it, and places it somewhere in the brain until a valid configuration is found. Panel C shows a valid example of a healthy mask (green) in the same brain as the tumor. These two different annotations are provided for the training set. The algorithms are supposed to inpaint both the masked tumor as well as the masked healthy area.
• Figure 4: Ranking Scheme of the challenge. First, a case-based ranking is calculated for every metric, i.e., for each test case $c_j, j=1,\cdots,N_c$, the performance $m_k(p_i,c_j)$ is calculated for each participating team $p_i, i=1,...,N_p$ and for each metric $m_k\in \{\text{\ac{RMSE}}, \text{\ac{PSNR}}, \text{\ac{SSIM}}\}$. Based on $m_k(p_i,c_j)$, a metric- and test-case specific rank $r_{j,k}(p_i)$ is calculated for each participating team. If $m_k(p_i,c_j) = \texttt{NA}$, $r_{j,k}(p_i)$ is set to the last rank. A metric-specific rank $r_k(p_i)$ is determined by aggregating the ranks over all cases. Second, the final rank for each participating team $r(p_i)$ is calculated over all metric-specific ranks.
• Figure 5: Violin plots showing the individual performance of participating teams. Results are shown for (a) SSIM, (b) RMSE, and (c) PSNR. Median performance is indicated by a thick horizontal line, mean performance by a rhombus. The upper and lower border of the boxplots illustrate the first and third quartile. The density of individual metric scores is shown by the violin plot.