Calibrated Self-supervised Vision Transformers Improve Intracranial Arterial Calcification Segmentation from Clinical CT Head Scans

TL;DR

This study addresses automated intracranial arterial calcification (IAC) segmentation on routine clinical CT head scans, a task challenged by real-world data heterogeneity. It introduces a calibrated, self-supervised approach using 3D Vision Transformers (ViTs) trained with Masked Autoencoder (MAE) pre-training, followed by supervised fine-tuning with multiple decoders. The method achieves a Dice score of 65.1 (95% CI [62.0;68.4]), beating a strong nnU-Net baseline by 3.2 Dice points, and demonstrates greater robustness to slice thickness and improved clinical risk-group classification by 46%. These results highlight the potential of MAE-pre-trained ViTs for scalable, real-world 3D medical image segmentation and large-scale IAC burden assessment.

Abstract

Vision Transformers (ViTs) have gained significant popularity in the natural image domain but have been less successful in 3D medical image segmentation. Nevertheless, 3D ViTs are particularly interesting for large medical imaging volumes due to their efficient self-supervised training within the masked autoencoder (MAE) framework, which enables the use of imaging data without the need for expensive manual annotations. Intracranial arterial calcification (IAC) is an imaging biomarker visible on routinely acquired CT scans linked to neurovascular diseases such as stroke and dementia, and automated IAC quantification could enable their large-scale risk assessment. We pre-train ViTs with MAE and fine-tune them for IAC segmentation for the first time. To develop our models, we use highly heterogeneous data from a large clinical trial, the third International Stroke Trial (IST-3). We evaluate key aspects of MAE pre-trained ViTs in IAC segmentation, and analyse the clinical implications. We show: 1) our calibrated self-supervised ViT beats a strong supervised nnU-Net baseline by 3.2 Dice points, 2) low patch sizes are crucial for ViTs for IAC segmentation and interpolation upsampling with regular convolutions is preferable to transposed convolutions for ViT-based models, and 3) our ViTs increase robustness to higher slice thicknesses and improve risk group classification in a clinical scenario by 46%. Our code is available online.

Figures (6)

• Figure 1: RoI location around the cavernous sinus of the internal carotid artery (left) and selected slices from the RoI (right; lower slices are closer to the neck, upper slices are closer to the skull vault). Intracranial arterial calcification in red in the right column. Schematic head created in BioRender, BioRender.com/b4yd3yp.
• Figure 2: Masked Autoencoder pre-training for IAC segmentation.
• Figure 3: Pre-trained vit fine-tuning with segmentation decoder variants.
• Figure 4: Bland-Altman plots for the IAC volume (in mm$^3$) predicted by the pre-trained ViTiac-S model compared to the manual annotation before and after calibration.
• Figure 5: Mean Dice↑ of the segmentation decoders under varying patch sizes and MAE pre-trained encoder sizes.