3DGR-CT: Sparse-View CT Reconstruction with a 3D Gaussian Representation

TL;DR

Sparse-view CT offers lower radiation dose but elevates noise and artifacts. The paper introduces 3DGR-CT, a 3D Gaussian representation for CT volumes that integrates FBP-guided initialization with a differentiable CT projector, enabling end-to-end optimization and faster convergence than implicit neural representations. Through a two-stage optimization, adaptive density control, and a CUDA-accelerated voxelization pipeline, 3DGR-CT achieves superior reconstruction quality across diverse anatomies and demonstrates potential for real-time physical simulation. This approach improves generalization across acquisition setups without extensive retraining, suggesting practical impact for safer, high-quality low-dose CT imaging and related clinical workflows.

Abstract

Sparse-view computed tomography (CT) reduces radiation exposure by acquiring fewer projections, making it a valuable tool in clinical scenarios where low-dose radiation is essential. However, this often results in increased noise and artifacts due to limited data. In this paper we propose a novel 3D Gaussian representation (3DGR) based method for sparse-view CT reconstruction. Inspired by recent success in novel view synthesis driven by 3D Gaussian splatting, we leverage the efficiency and expressiveness of 3D Gaussian representation as an alternative to implicit neural representation. To unleash the potential of 3DGR for CT imaging scenario, we propose two key innovations: (i) FBP-image-guided Guassian initialization and (ii) efficient integration with a differentiable CT projector. Extensive experiments and ablations on diverse datasets demonstrate the proposed 3DGR-CT consistently outperforms state-of-the-art counterpart methods, achieving higher reconstruction accuracy with faster convergence. Furthermore, we showcase the potential of 3DGR-CT for real-time physical simulation, which holds important clinical applications while challenging for implicit neural representations.

Figures (13)

• Figure 1: Comparison of 3D Gaussian Representation (3DGR) and Implicit Neural Representation (INR). (a) Prior-Informed Initialization: Our FBP-image guided initialization technique effectively avoids placing Gaussians in void regions and allocates their density (corresponds to modeling capacity) according to region complexity. Computations for each voxel only involve nearby Gaussians. In contrast, INR need to compute the forward process of a fully connected network many times. (b) Adaptive Density: Neural networks treat each coordinate equally, while the density of 3D Gaussians can be adaptively adjusted during optimization through cloning and splitting operations. This dynamic allocation of modeling capacity allows for superior reconstruction of fine details.
• Figure 2: Our method achieves similar quantitative performance with state-of-the-art INR based method with only half of its time, and continue to arrive at a much better visual and quantitative result given more time.
• Figure 3: Overview of pipeline: Initially, projection data is acquired from various viewpoints and subsequently processed using Filtered Back Projection (FBP) to yield the FBP-reconstructed image. This reconstructed image is then utilized as a prior for initializing the 3D Gaussians. During the training phase, these 3D Gaussians are discretized into a volumetric image, which then undergoes a forward projection process. We compare the difference between these projections and the ground truth measurements, and use the gradients to update parameters of Gaussians. Ultimately, the interplay of the gradient and the Gaussians' parameters dictates the density control of the Gaussians.
• Figure 4: Visualization of reconstructed images. The window level and width are optimized for visualizing brain tissue, while other anatomical structures are displayed using standard CT value ranges. Observe that NeRP, without prior information, generates blurry reconstructions lacking intricate details. NAF, on the other hand, is prone to introducing additional noise and artifacts. Compared to INRs, our proposed 3DGR provides cleaner results in empty regions and better high-frequency details like the airways. The visual difference is especially prominent in the abdomen, chest, and head regions.
• Figure 5: Zoomed-in comparison of critical anatomical structures.