3D Spine Shape Estimation from Single 2D DXA

TL;DR

The paper tackles reconstructing patient-specific 3D spine shape from a single 2D DXA image by learning to predict two orthogonal spine projections. It employs a ResNet50-based encoder with a Bottleneck Transformer to regress coronal and sagittal spine curves using paired DXA–MRI data, enabling 3D reconstruction from the two projections. Quantitative results on UK Biobank data show strong 2D curve accuracy and 3D spine reconstruction quality, with a 3D spine curve deviation of about 3.12 mm and 3D IoU around 83.8%. This approach reduces radiation exposure while providing 3D spine assessments, offering a potential tool for efficient scoliosis screening and monitoring.

Abstract

Scoliosis is traditionally assessed based solely on 2D lateral deviations, but recent studies have also revealed the importance of other imaging planes in understanding the deformation of the spine. Consequently, extracting the spinal geometry in 3D would help quantify these spinal deformations and aid diagnosis. In this study, we propose an automated general framework to estimate the 3D spine shape from 2D DXA scans. We achieve this by explicitly predicting the sagittal view of the spine from the DXA scan. Using these two orthogonal projections of the spine (coronal in DXA, and sagittal from the prediction), we are able to describe the 3D shape of the spine. The prediction is learnt from over 30k paired images of DXA and MRI scans. We assess the performance of the method on a held out test set, and achieve high accuracy.

Figures (8)

• Figure 1: Inference. Given a DXA scan, the model predicts the coronal and sagittal projections of the 3D spine. Once these two orthogonal views of the spine are obtained, the 3D spine can be reconstructed. A visualisation of the rotating spine is given at the website https://www.robots.ox.ac.uk/ vgg/research/dxa-to-3d.
• Figure 2: DXA to MRI two-Stage Alignment. The two scans are iteratively aligned using a three parameter planar transformation. From left to right: DXA scan; original coronal projection of MRI scan (not-aligned to the DXA); overlay of MRI aligned to DXA after the image-level alignment first stage; overlay of segmented spines after the first stage; overlay of spine segmentation after the spine-level alignment second stage.
• Figure 3: Model Learning. The regression model is learnt from pairs of aligned DXA and MRI scans. The regression targets are the DXA curve, and the sagittal curve (projected from the 3D MRI spine). The alignment for the sagittal curve to DXA is obtained from the alignment of the coronal projection of the 3D MRI to the DXA. Six curves are regressed: the centerline of the spine as well as the left and right boundaries of the segmentation, for both the coronal and sagittal views.
• Figure 4: Image-Based Regression of Coronal and Sagittal Spine Curves. We use a ResNet50, pre-trained on ImageNet-21k, with a transformer layer to regress the spine curves $(x_{(1,2,3)}(z),y_{(1,2,3)}(z)), z \in [1,209]$ for left, center and right curves. The feature map extracted from ResNet50 are of resolution 7 x 7 x 2048, each vector feature from ResNet50 (49 x 2048) is used as input into a transformer layer. The model regresses the 6 curves (209 x 6) where we have 6 vectors for the 6 output spine curves, of dimension 209. Detailed Architecture in Appendix \ref{['Appendix:Implementation Details and Ablation']} Figure \ref{['fig_supp:Regression_Architecture']}.
• Figure 5: Qualitative Results of DXA to MRI Sagittal Curve Generation on the Test Set. We show from left to right, the input DXA, the ground-truth sagittal curves, predicted sagittal curves, ground-truth spine mask, predicted spine mask overlayed on MRI sagittal slice (n=112), the ground-truth 3D spine, and its prediction using Gaussian rendering. The severity of the scoliosis increases from top to bottom.