IXGS-Intraoperative 3D Reconstruction from Sparse, Arbitrarily Posed Real X-rays

TL;DR

IXGS addresses the challenge of reconstructing 3D spinal anatomy from sparse, arbitrarily posed intraoperative X-rays by extending the $R^{2}$-Gaussian splatting framework to non-circular acquisition and eliminating the need for anatomy-specific pretraining. A key novelty is anatomy-guided radiographic standardization, implemented via a style-transfer network trained on paired ex-vivo real X-rays and synthetic DRRs, producing $I_{ ext{ST}}$ that improves consistency and highlights bone structures for the subsequent Gaussian optimization. The pipeline outputs a volumetric density $V$ through a differentiable voxelizer and provides 3D renderings and multiplanar views to support navigation. Expert surgeon evaluation on ex-vivo data demonstrates usability improvements when using roughly 20–30 views, with quantitative PSNR/SSIM showing a trade-off relative to idealized baselines but a clear benefit from standardization. Importantly, IXGS achieves instance-based 3D reconstruction from arbitrary sparse-view X-rays without anatomy-specific pretraining, and code will be released to facilitate broader adoption.

Abstract

Spine surgery is a high-risk intervention demanding precise execution, often supported by image-based navigation systems. Recently, supervised learning approaches have gained attention for reconstructing 3D spinal anatomy from sparse fluoroscopic data, significantly reducing reliance on radiation-intensive 3D imaging systems. However, these methods typically require large amounts of annotated training data and may struggle to generalize across varying patient anatomies or imaging conditions. Instance-learning approaches like Gaussian splatting could offer an alternative by avoiding extensive annotation requirements. While Gaussian splatting has shown promise for novel view synthesis, its application to sparse, arbitrarily posed real intraoperative X-rays has remained largely unexplored. This work addresses this limitation by extending the $R^2$-Gaussian splatting framework to reconstruct anatomically consistent 3D volumes under these challenging conditions. We introduce an anatomy-guided radiographic standardization step using style transfer, improving visual consistency across views, and enhancing reconstruction quality. Notably, our framework requires no pretraining, making it inherently adaptable to new patients and anatomies. We evaluated our approach using an ex-vivo dataset. Expert surgical evaluation confirmed the clinical utility of the 3D reconstructions for navigation, especially when using 20 to 30 views, and highlighted the standardization's benefit for anatomical clarity. Benchmarking via quantitative 2D metrics (PSNR/SSIM) confirmed performance trade-offs compared to idealized settings, but also validated the improvement gained from standardization over raw inputs. This work demonstrates the feasibility of instance-based volumetric reconstruction from arbitrary sparse-view X-rays, advancing intraoperative 3D imaging for surgical navigation.

Figures (7)

• Figure 1: Comparison between conventional circular acquisition paths, typical for CBCT/CT imaging or idealized synthetic DRR generation (left), and the irregular, arbitrary acquisition poses representative of real intraoperative settings (right).
• Figure 2: Pipeline Overview. The training of Pix2Pix (blue) uses paired real X-rays ($\mathbf{I}_{\text{real}}$) and synthetic DRRs ($\mathbf{I}_{\text{DRR}}$) to learn style transfer. During inference (green), real X-rays with calibration information are converted to style-transferred images ($\mathbf{I}_{\text{ST}}$). These images, along with their poses, are passed to the Gaussian splatting network, which outputs 3D reconstructions. The resulting volume can be visualized as slices or rendered from arbitrary viewpoints (orange).
• Figure 3: Comparison of 3D reconstructions of the lumbar spine (axial, coronal, and sagittal slices). Each block shows axial, coronal, and sagittal slices of the reconstructed volume, overlaid with alpha-blended masks of the segmented lumbar spine from the ground truth CT for better accuracy assessment. The columns compare reconstructions using 25 views (left) and 50 views (right). The top row shows reconstructions from $\mathbf{I}_{\text{real}}$ baseline, while the bottom row displays reconstructions from $\mathbf{I}_{\text{ST}}$ (our approach).
• Figure 4: Comparison of slices from reconstructed volumes using different inputs and methods. From left to right: $\mathbf{V}_{\text{CT}}$: Ground Truth CT volume, $\mathbf{V}_{\text{circ}}$: Reconstruction from 50 synthetic DRRs generated from circular acquisition, $\mathbf{V}_{\text{real}}$: Reconstruction from 50 X-rays generated from arbitrary poses, $\mathbf{V}_{\text{ST}}$ (our approach): Reconstruction from 50 style-transferred X-rays.
• Figure 5: Comparison of 3D reconstructions of the lumbar spine. Each block shows AP, lateral, and isometric views of the reconstructed volume. The columns compare reconstructions using 25 views (left) and 50 views (right). The top row displays reconstructions from $\mathbf{I}_{\text{real}}$ baseline, while the bottom row shows reconstructions from $\mathbf{I}_{\text{ST}}$ (our approach).