3D Path Planning for Robot-assisted Vertebroplasty from Arbitrary Bi-plane X-ray via Differentiable Rendering

TL;DR

Vertebral augmentation often lacks preoperative CT for planning. The authors introduce Spine-DART, a differentiable rendering framework that reconstructs 3D vertebrae and plans transpedicular paths from two arbitrary X-ray views using a vertebral SSM and a learned similarity loss. The method blends DL-based X-ray annotation, differentiable rendering, and gradient-based optimization to deliver anatomically plausible reconstructions and automatic planning without fixed imaging geometries. Results show competitive reconstruction performance against state-of-the-art models and improved clinical viability over a 2D-GeoPlan baseline, across synthetic and real X-ray data, enabling CT-free, flexible intraoperative guidance for vertebroplasty.

Abstract

Robotic systems are transforming image-guided interventions by enhancing accuracy and minimizing radiation exposure. A significant challenge in robotic assistance lies in surgical path planning, which often relies on the registration of intraoperative 2D images with preoperative 3D CT scans. This requirement can be burdensome and costly, particularly in procedures like vertebroplasty, where preoperative CT scans are not routinely performed. To address this issue, we introduce a differentiable rendering-based framework for 3D transpedicular path planning utilizing bi-planar 2D X-rays. Our method integrates differentiable rendering with a vertebral atlas generated through a Statistical Shape Model (SSM) and employs a learned similarity loss to refine the SSM shape and pose dynamically, independent of fixed imaging geometries. We evaluated our framework in two stages: first, through vertebral reconstruction from orthogonal X-rays for benchmarking, and second, via clinician-in-the-loop path planning using arbitrary-view X-rays. Our results indicate that our method outperformed a normalized cross-correlation baseline in reconstruction metrics (DICE: 0.75 vs. 0.65) and achieved comparable performance to the state-of-the-art model ReVerteR (DICE: 0.77), while maintaining generalization to arbitrary views. Success rates for bipedicular planning reached 82% with synthetic data and 75% with cadaver data, exceeding the 66% and 31% rates of a 2D-to-3D baseline, respectively. In conclusion, our framework facilitates versatile, CT-free 3D path planning for robot-assisted vertebroplasty, effectively accommodating real-world imaging diversity without the need for preoperative CT scans.

Figures (10)

• Figure 1: Proposed pipeline overview: Two SWIN Transformers analyze X-ray images to detect and segment vertebrae, identifying landmarks to initialize the SSM's pose. Gaussian splatting is used to render SSM projections in optimization steps. A multi-view Vision Transformer (ViT) calculates a similarity loss from these projections and original X-rays, refining the SSM's pose and shape for accurate 3D vertebral reconstruction with consistent point correspondences.
• Figure 2: 3D vertebral landmarks used for initialization of the SSM 3D pose.
• Figure 3: Top: segmentation performance metrics for synthetic DRRs and real X-rays. Right: qualitative vertebra segmentation result on a real X-ray.
• Figure 4: Qualitative results of our landmark detection model. Predictions (circles) and ground truths (stars) overlaid on real X-ray images.
• Figure 5: Landmark detection results across different domains. We report the mean Euclidean distance (mm) versus confidence percentile for orthogonal (AP/LAT) (left), random (middle), and real X-ray views (right).