Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Stable Optimization for Large Vision Model Based Deep Image Prior in Cone-Beam CT Reconstruction

Minghui Wu, Yangdi Xu, Yingying Xu, Guangwei Wu, Qingqing Chen, Hongxiang Lin

TL;DR

This work tackles sparse-view CBCT reconstruction with minimal data by marrying a forward-model-free DIP framework to a Large Vision Model backbone. It introduces a multi-scale perceptual loss (MSPL) derived from a frozen VGG-11 encoder and an adaptive reweighting one-shot optimization to stabilize training and simultaneously update MSPL weights and network parameters. The loss combines a reconstruction term, MSPL, and a TV penalty, formalized as $L = L_1 + \alpha L_2 + \beta \text{TV}(\theta)$ with $L_1(\theta) = \|\hat{X}-X_\theta\|_2^2$ and MSPL defined through feature differences across multiple layers. On SPARE and Walnut datasets, the method achieves significant gains in PSNR/SSIM and reduces streak artifacts, validated by radiologists, indicating practical potential for high-quality CBCT from limited projections. This approach offers a path toward data-efficient tomography with broad applicability to other imaging modalities.

Abstract

Large Vision Model (LVM) has recently demonstrated great potential for medical imaging tasks, potentially enabling image enhancement for sparse-view Cone-Beam Computed Tomography (CBCT), despite requiring a substantial amount of data for training. Meanwhile, Deep Image Prior (DIP) effectively guides an untrained neural network to generate high-quality CBCT images without any training data. However, the original DIP method relies on a well-defined forward model and a large-capacity backbone network, which is notoriously difficult to converge. In this paper, we propose a stable optimization method for the forward-model-free, LVM-based DIP model for sparse-view CBCT. Our approach consists of two main characteristics: (1) multi-scale perceptual loss (MSPL) which measures the similarity of perceptual features between the reference and output images at multiple resolutions without the need for any forward model, and (2) a reweighting mechanism that stabilizes the iteration trajectory of MSPL. One shot optimization is used to simultaneously and stably reweight MSPL and optimize LVM. We evaluate our approach on two publicly available datasets: SPARE and Walnut. The results show significant improvements in both image quality metrics and visualization that demonstrates reduced streak artifacts. The source code is available upon request.

Stable Optimization for Large Vision Model Based Deep Image Prior in Cone-Beam CT Reconstruction

TL;DR

This work tackles sparse-view CBCT reconstruction with minimal data by marrying a forward-model-free DIP framework to a Large Vision Model backbone. It introduces a multi-scale perceptual loss (MSPL) derived from a frozen VGG-11 encoder and an adaptive reweighting one-shot optimization to stabilize training and simultaneously update MSPL weights and network parameters. The loss combines a reconstruction term, MSPL, and a TV penalty, formalized as with and MSPL defined through feature differences across multiple layers. On SPARE and Walnut datasets, the method achieves significant gains in PSNR/SSIM and reduces streak artifacts, validated by radiologists, indicating practical potential for high-quality CBCT from limited projections. This approach offers a path toward data-efficient tomography with broad applicability to other imaging modalities.

Abstract

Large Vision Model (LVM) has recently demonstrated great potential for medical imaging tasks, potentially enabling image enhancement for sparse-view Cone-Beam Computed Tomography (CBCT), despite requiring a substantial amount of data for training. Meanwhile, Deep Image Prior (DIP) effectively guides an untrained neural network to generate high-quality CBCT images without any training data. However, the original DIP method relies on a well-defined forward model and a large-capacity backbone network, which is notoriously difficult to converge. In this paper, we propose a stable optimization method for the forward-model-free, LVM-based DIP model for sparse-view CBCT. Our approach consists of two main characteristics: (1) multi-scale perceptual loss (MSPL) which measures the similarity of perceptual features between the reference and output images at multiple resolutions without the need for any forward model, and (2) a reweighting mechanism that stabilizes the iteration trajectory of MSPL. One shot optimization is used to simultaneously and stably reweight MSPL and optimize LVM. We evaluate our approach on two publicly available datasets: SPARE and Walnut. The results show significant improvements in both image quality metrics and visualization that demonstrates reduced streak artifacts. The source code is available upon request.
Paper Structure (7 sections, 3 equations, 4 figures, 1 table)

This paper contains 7 sections, 3 equations, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Method
  3. Model
  4. Loss Function
  5. Reweighting Mechanism for Stable Optimization
  6. Experiment and Results
  7. Discussion and Conclusion

Figures (4)

  • Figure 1: The framework of the proposed forward-model-free, LVM-based DIP with MSPL for sparse-view CBCT reconstruction. Texts with red highlight the learnable variables including loss and network weights.
  • Figure 2: The flow chart of the adaptive reweighting one shot optimization algorithm.
  • Figure 3: Visualization of the sagittal views by (a) FDK, (b) SIRT, (c) DIP with the 3D U-Net backbone, (d) DIP with the UNETR backbone, (e) Our proposed approach w/o TV, (f) Our proposed approach w/ TV (g) full-view Ground Truth (GT) on the $107$th profile of the last subject in the SPARE dataset. Global image, Zoomed image, and Error map are illustrated.
  • Figure 4: Ablation studies for our proposed method. (a) Three different weighting strategies for MSPL with $\beta=0$: w/o MSPL ($\mathbf{w}=0$), fixing weights ($\mathbf{w}=1$), and automatic reweighting strategy (Reweight $\mathbf{w}$); (b) Convergence analysis w/ and w/o the TV penalty by controlling $\beta=1$ and $0$ while using Reweight $\mathbf{w}$; (c) Performance analysis (evaluated by PSNR and SSIM) for three downsampling operations: Resampling method (RM), center-clipping method (CCM), and DSConv; (d) Evaluation of representative features corresponding to activation layers by means of SSPL, and MSPL with $\mathbf{w}=1$ and Reweight $\mathbf{w}$.