Swap-Net: A Memory-Efficient 2.5D Network for Sparse-View 3D Cone Beam CT Reconstruction

TL;DR

This work tackles sparse-view 3D CBCT reconstruction by introducing Swap-Net, a memory-efficient 2.5D CNN that uses axes-swapping to perform axis-wise 2D convolutions while producing full 3D outputs from FBP inputs. The model avoids full 3D convolutions yet captures cross-dimensional context, enabling end-to-end training to refine artifact-prone volumes under AWGN and non-ideal physics. Swap-Net outperforms classical methods and 2D/3D DL baselines in SNR and SSIM while using far fewer parameters and modest GPU memory, aided by a three-block cascade and an axis-swapping strategy that enforces global 3D consistency. The approach has strong implications for practical 3D CBCT and may extend to other 3D imaging tasks requiring memory-efficient, high-quality reconstruction from limited projections.

Abstract

Reconstructing 3D cone beam computed tomography (CBCT) images from a limited set of projections is an important inverse problem in many imaging applications from medicine to inertial confinement fusion (ICF). The performance of traditional methods such as filtered back projection (FBP) and model-based regularization is sub-optimal when the number of available projections is limited. In the past decade, deep learning (DL) has gained great popularity for solving CT inverse problems. A typical DL-based method for CBCT image reconstruction is to learn an end-to-end mapping by training a 2D or 3D network. However, 2D networks fail to fully use global information. While 3D networks are desirable, they become impractical as image sizes increase because of the high memory cost. This paper proposes Swap-Net, a memory-efficient 2.5D network for sparse-view 3D CBCT image reconstruction. Swap-Net uses a sequence of novel axes-swapping operations to produce 3D volume reconstruction in an end-to-end fashion without using full 3D convolutions. Simulation results show that Swap-Net consistently outperforms baseline methods both quantitatively and qualitatively in terms of reducing artifacts and preserving details of complex hydrodynamic simulations of relevance to the ICF community.

Figures (11)

• Figure 1: Overview of the proposed Swap-Net framework for training an end-to-end deep mapping for 3D CBCT image reconstruction using ICF synthetic radiographs. The Swap-Net model ${\mathsf{R}}_{\bm{\theta }}$ is implemented as a customized architecture mapping the output of FBP to the desired ground-truth 3D images. The novel axes-swapping operation in Swap-Net allows it to efficiently conduct convolution across all dimensions. The whole network is trained end-to-end in a supervised fashion.
• Figure 2: The ICF models: (a) A typical double-shell ICF capsule containing Deuterium/Tritium, Beryllium tamper, Tungsten pusher, low density $\text{CH}_4$ foam, and Aluminum. (b) A simplified representation of a ICF implosion capsule containing Deuterium/Tritium (Ablator), Tungsten pusher (Tantalum), and low density $\text{CH}_4$ foam (Gas)merritt2019experimental.
• Figure 3: Central slices along each dimension of an exemplar 3D ICF object generated for an ICF double shell simulation in our dataset. The two materials that form the object, namely gas and metal, are labeled in each image. The images presented here were normalized by the mass attenuation factor to the range of [0, 2] for good visualization (same in the rest of the paper).
• Figure 4: Statistical summary of SNR values for different reconstruction methods evaluated on 2D slices along each dimension taken from our test set. Plots in the first and second row correspond to the the results with 4 projection views under AWGN and non-ideal physics including blur and scatter and non-white noise corruptions, respectively.
• Figure 5: Visual evaluation of Swap-Net and baseline methods on an exemplar ICF double shell test simulation with $4$ projection views under AWGN corruption. Each row shows the middle slice of the whole 3D object along $z$, $y$ and $x$ axes, respectively. The bottom part of each image provides the SNR and SSIM values and representative $2\times$ zoomed-in regions and their error maps with respect to the ground truth. Arrows in the zoomed-in plots highlight sharp edges that are well reconstructed using Swap-Net. Note the excellent quantitative and qualitative performance of Swap-Net for both artifact correction and detail preservation.