SPINEPS -- Automatic Whole Spine Segmentation of T2-weighted MR images using a Two-Phase Approach to Multi-class Semantic and Instance Segmentation

TL;DR

This study introduces an automatic approach to whole spine segmentation, including posterior elements, in arbitrary fields of view T2w sagittal MR images, enabling easy biomarker extraction, automatic localization of pathologies and degenerative diseases, and quantifying analyses as downstream research.

Abstract

Purpose. To present SPINEPS, an open-source deep learning approach for semantic and instance segmentation of 14 spinal structures (ten vertebra substructures, intervertebral discs, spinal cord, spinal canal, and sacrum) in whole body T2w MRI. Methods. During this HIPPA-compliant, retrospective study, we utilized the public SPIDER dataset (218 subjects, 63% female) and a subset of the German National Cohort (1423 subjects, mean age 53, 49% female) for training and evaluation. We combined CT and T2w segmentations to train models that segment 14 spinal structures in T2w sagittal scans both semantically and instance-wise. Performance evaluation metrics included Dice similarity coefficient, average symmetrical surface distance, panoptic quality, segmentation quality, and recognition quality. Statistical significance was assessed using the Wilcoxon signed-rank test. An in-house dataset was used to qualitatively evaluate out-of-distribution samples. Results. On the public dataset, our approach outperformed the baseline (instance-wise vertebra dice score 0.929 vs. 0.907, p-value<0.001). Training on auto-generated annotations and evaluating on manually corrected test data from the GNC yielded global dice scores of 0.900 for vertebrae, 0.960 for intervertebral discs, and 0.947 for the spinal canal. Incorporating the SPIDER dataset during training increased these scores to 0.920, 0.967, 0.958, respectively. Conclusions. The proposed segmentation approach offers robust segmentation of 14 spinal structures in T2w sagittal images, including the spinal cord, spinal canal, intervertebral discs, endplate, sacrum, and vertebrae. The approach yields both a semantic and instance mask as output, thus being easy to utilize. This marks the first publicly available algorithm for whole spine segmentation in sagittal T2w MR imaging.

Figures (5)

• Figure 1: Showcase of the three automated annotations and its resulting combined annotation, as a 2D segmentation overlay and a 3D snapshot. a) shows the segmentation made with the training data from Streckenbach et al. streckenbach_application_2022, b) the SpinalCordToolbox annotation, c) the annotations derived from translation, and d) is the combination of all three. We observed that the manual segmentation from Streckenbach et al. streckenbach_application_2022 is primarily block-shaped and incomplete, while the translated annotations often segmented too many voxels around the vertebra corpus.
• Figure 2: The data flow of our proposed method of inference on new T2w sagittal scans. The semantic model segments 14 different spine structures, regardless of field of view. Then, cutouts are made from that segmentation, which are fed into the instance model. The results are predictions for the individual vertebrae, which are fused together for the vertebra instance mask. Then, using the first segmentation, we zero out everything in the vertebra mask that is not present in the subregion mask. Finally, we match intervertebral discs and endplate instances based on a center of mass analysis.
• Figure 3: Given the semantic segmentation, we create cutouts of exact same size. Each of those cutouts (colored boxes) is fed into the instance model. a), b), and c) show the first three predictions predictions of a semantic input. The instance model always predicts the center vertebra of the cutout (green), as well as the one above (red) and below (blue), if visible. Therefore, assuming no erroneous behavior, we get three predictions for all inner vertebrae and two for the outer ones. For example, the second to last vertebra in the figure is predicted thrice, once in each of the three predictions (red, green, blue, from left to right). The combination of all cutout predictions are combined into a vertebra instance mask d), labeling each vertebra instance uniquely (different colors).
• Figure 4: Flow diagram for subject exclusion from top to bottom for the different datasets. We only excluded subjects from the German National Cohort (GNC) because some of the automated annotation generation approaches failed for some subjects.
• Figure 5: Example subjects where the baseline (a) produces a typical found error: mixing different instance labels. Our approach (b) is very close to the reference annotation (c). This type of error the baseline made did not occur with our approach.