SegmentAnyBone: A Universal Model that Segments Any Bone at Any Location on MRI

TL;DR

This work addresses the lack of universal MRI bone segmentation across body locations and sequences by introducing SegmentAnyBone, a SAM-based framework enhanced with a two-branch architecture that includes a 2D SAM-guided pathway and a 3D depth-attention module. The method employs parameter-efficient fine-tuning via adapters, a hybrid prompting strategy to support automatic and prompt-based segmentation, and a learnable fusion gate to integrate 3D information, achieving robust generalization to unseen locations, sequences, and external data. A new multi-location MRI bone dataset is created and annotated, with extensions to non-T1 sequences and an external Lumbar-Spine dataset for external validation. The results show state-of-the-art performance in cross-location MRI bone segmentation, particularly outperforming 2D baselines and many 3D models, while also enabling interactive corrections through prompts; these findings imply a practical, scalable tool for radiological workflow and downstream musculoskeletal analysis.

Abstract

Magnetic Resonance Imaging (MRI) is pivotal in radiology, offering non-invasive and high-quality insights into the human body. Precise segmentation of MRIs into different organs and tissues would be highly beneficial since it would allow for a higher level of understanding of the image content and enable important measurements, which are essential for accurate diagnosis and effective treatment planning. Specifically, segmenting bones in MRI would allow for more quantitative assessments of musculoskeletal conditions, while such assessments are largely absent in current radiological practice. The difficulty of bone MRI segmentation is illustrated by the fact that limited algorithms are publicly available for use, and those contained in the literature typically address a specific anatomic area. In our study, we propose a versatile, publicly available deep-learning model for bone segmentation in MRI across multiple standard MRI locations. The proposed model can operate in two modes: fully automated segmentation and prompt-based segmentation. Our contributions include (1) collecting and annotating a new MRI dataset across various MRI protocols, encompassing over 300 annotated volumes and 8485 annotated slices across diverse anatomic regions; (2) investigating several standard network architectures and strategies for automated segmentation; (3) introducing SegmentAnyBone, an innovative foundational model-based approach that extends Segment Anything Model (SAM); (4) comparative analysis of our algorithm and previous approaches; and (5) generalization analysis of our algorithm across different anatomical locations and MRI sequences, as well as an external dataset. We publicly release our model at https://github.com/mazurowski-lab/SegmentAnyBone.

Figures (15)

• Figure 1: Dataset visualization and composition. The training dataset contains 17 locations: Humerus, Thoracic Spine, Lumbar Spine, Forearm, Pelvis, Hand, Lower Leg, Ankle, Shoulder, Chest, Arm, Elbow, Hip, Wrist, Thigh, Knee, and Foot. For Humerus, Shoulder, Lower Leg, and Knee, examples of MRI slices from three different views, Axial (AX), Coronal (COR), and Sagittal (SAG), are shown.
• Figure 2: Comparative demonstration of alignment quality. We enhanced the image contrast for better visualization.
• Figure 3: An overview of the model pipeline. Our model consists of the original SAM branch along with an additional 3D attention branch. These two branches are combined through a learnable gate. In the training phase, only the Adapters within the attention blocks is updated. Additionally, our approach employs a hybrid prompting technique that involves either providing specific prompts or performing automatic segmentation.
• Figure 4: Visualization of the segmentation performance on the test set. The average intersection over union (IoU) and dice coefficient (DSC) scores are listed with green arrows showing the 95% confidence interval.
• Figure 5: Visualization of the automatic segmentation performance across various sequences from the same exam. The first row shows one exam with predicted performance on T1, T2, and STIR, and the second row shows the predicted performance for T1, T2, and Proton Density TSE sequences from a Knee exam.