X-LRM: X-ray Large Reconstruction Model for Extremely Sparse-View Computed Tomography Recovery in One Second

TL;DR

This work tackles extremely sparse-view CT reconstruction by introducing X-LRM, a Transformer-based feedforward model comprising an MLP-based X-former tokenizer and a triplane-based X-triplane decoder that produces a 3D radiodensity field. It leverages Torso-16K, a large-scale CT reconstruction dataset, to enable scalable training and robust generalization across varying numbers of input projections. Empirically, X-LRM achieves about 1.5 dB PSNR improvement over the state-of-the-art 3D feedforward methods and runs roughly 27× faster, while maintaining flexibility to handle 6–10 views without retraining. The approach also shows practical value in downstream segmentation tasks, suggesting strong clinical potential for fast and accurate CT reconstruction from extremely sparse projections.

Abstract

Sparse-view 3D CT reconstruction aims to recover volumetric structures from a limited number of 2D X-ray projections. Existing feedforward methods are constrained by the limited capacity of CNN-based architectures and the scarcity of large-scale training datasets. In this paper, we propose an X-ray Large Reconstruction Model (X-LRM) for extremely sparse-view (<10 views) CT reconstruction. X-LRM consists of two key components: X-former and X-triplane. Our X-former can handle an arbitrary number of input views using an MLP-based image tokenizer and a Transformer-based encoder. The output tokens are then upsampled into our X-triplane representation, which models the 3D radiodensity as an implicit neural field. To support the training of X-LRM, we introduce Torso-16K, a large-scale dataset comprising over 16K volume-projection pairs of various torso organs. Extensive experiments demonstrate that X-LRM outperforms the state-of-the-art method by 1.5 dB and achieves 27x faster speed and better flexibility. Furthermore, the downstream evaluation of lung segmentation tasks also suggests the practical value of our approach. Our code, pre-trained models, and dataset will be released at https://github.com/caiyuanhao1998/X-LRM

Figures (7)

• Figure 1: Our X-LRM outperforms previous state-of-the-art 3D feedforward methods in terms of quality and efficiency, including DIF-Net difnet, DIF-Gaussian dif_gs, and C$^2$-RV c2rv. Our collected CT dataset, Torso-16K, is over 18$\times$ larger than previous benchmarks: LUNA16 setio2017validation, ToothFairy cipriano2022deep, and AAPM-Myo mccollough2016tu.
• Figure 1: The statistics of our collected Torso-16K benchmark. Torso-16K integrates ten public datasets covering major anatomical regions in different clinical applications.
• Figure 2: The overall architecture of X-LRM: (a) We collect Torso-16K, the largest CT reconstruction dataset (\ref{['exp:setup']}) to the best of our knowledge. (b) Our X-Former features an image tokenizer and encoder, designed to process a variable number of input views (\ref{['method:x_former']}). (c) Our X-Triplane includes a triplane decoder followed by our implicit neural field, directly predicting the 3D CT volume $\hat{\mathbf{U}}$ (\ref{['method:x_triplane']}).
• Figure 3: Example volumes and X-ray projectionis in Torso-16K dataset.
• Figure 4: Qualitative results of feedforward methods. From top to bottom: 10-view axial, 8-view coronal, and 6-view sagittal slices.