DeepSparse: A Foundation Model for Sparse-View CBCT Reconstruction

TL;DR

DeepSparse tackles the challenge of reconstructing 3D CBCT volumes from sparse X-ray projections to reduce radiation exposure. It introduces DiCE, a reconstruction framework that fuses multi-scale 2D features and back-projected 3D features with cross-scale embedding, and HyViP, a hybrid-view pretraining scheme that leverages both sparse and dense projections to improve generalization. A two-step finetuning process adapts the pretrained model to new datasets and view configurations, incorporating a denoising layer to align sparse-view 3D features with dense-view priors. Experimental results on chest and knee CBCT datasets show superior reconstruction quality (PSNR/SSIM) and substantially faster inference with far fewer parameters than previous INR-based approaches, indicating strong potential for safer, more efficient CBCT imaging in clinical settings.

Abstract

Cone-beam computed tomography (CBCT) is a critical 3D imaging technology in the medical field, while the high radiation exposure required for high-quality imaging raises significant concerns, particularly for vulnerable populations. Sparse-view reconstruction reduces radiation by using fewer X-ray projections while maintaining image quality, yet existing methods face challenges such as high computational demands and poor generalizability to different datasets. To overcome these limitations, we propose DeepSparse, the first foundation model for sparse-view CBCT reconstruction, featuring DiCE (Dual-Dimensional Cross-Scale Embedding), a novel network that integrates multi-view 2D features and multi-scale 3D features. Additionally, we introduce the HyViP (Hybrid View Sampling Pretraining) framework, which pretrains the model on large datasets with both sparse-view and dense-view projections, and a two-step finetuning strategy to adapt and refine the model for new datasets. Extensive experiments and ablation studies demonstrate that our proposed DeepSparse achieves superior reconstruction quality compared to state-of-the-art methods, paving the way for safer and more efficient CBCT imaging.

Figures (7)

• Figure 1: (a) During the CBCT scanning, the X-ray source will emit cone-shaped beams, and the measurement is a 2D projection at each view. (b) Our DeepSparse is pretrained on a large-scale CT dataset, covering various body organs with different projection parameters. DeepSparse can be further finetuned on target datasets to achieve the state-of-the-art reconstruction performance.
• Figure 2: Overview of the reconstruction framework DiCE. The 2D encoder extracts multi-scale semantic features from sparse-view 2D projections. At each scale, these multi-view features are back-projected into a low-resolution volumetric space to generate 3D features. The 3D decoder then aggregates the multi-scale 3D features to produce an enhanced 3D representation.
• Figure 3: Attenuation prediction. For a sampled 3D point, we obtain its multi-scale pixel-aligned features from multi-view multi-scale 2D features by projection, interpolation, and applying max-pooling. Similarly, we obtain the voxel-aligned features through interpolation. Then, these features are concatenated and passed into the point decoder, predicting the corresponding attenuation coefficient for the point.
• Figure 4: Overview of HyViP pretraining framework. In each training iteration, we randomly select an $N$ and sample $N$-view sparse projections and $N_\text{max}$-view dense projections, which are then used to generate multi-view multi-scale 2D features and 3D representation, respectively.
• Figure 5: Overview of finetuning step-2. For sparse inputs, additional denoise layers ($\sigma^\text{denoise}_i$) are introduced to refine the quantized 3D features. For dense inputs, we stop the gradient propagation and use the quantized 3D features as a supervision to compute the denoise loss $\mathcal{L}_\text{denoise}$. Finally, only features (i.e., $\mathcal{F}_i$ and $F^\text{3D}$) generated from sparse projections are used to predict the attenuation coefficients.