Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

R$^2$-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction

Ruyi Zha, Tao Jun Lin, Yuanhao Cai, Jiwen Cao, Yanhao Zhang, Hongdong Li

TL;DR

R-Gaussian, the first 3DGS-based framework for sparse-view tomographic reconstruction, is introduced, which outperforms state-of-the-art approaches in accuracy and efficiency and delivers high-quality results in 4 minutes.

Abstract

3D Gaussian splatting (3DGS) has shown promising results in image rendering and surface reconstruction. However, its potential in volumetric reconstruction tasks, such as X-ray computed tomography, remains under-explored. This paper introduces R$^2$-Gaussian, the first 3DGS-based framework for sparse-view tomographic reconstruction. By carefully deriving X-ray rasterization functions, we discover a previously unknown integration bias in the standard 3DGS formulation, which hampers accurate volume retrieval. To address this issue, we propose a novel rectification technique via refactoring the projection from 3D to 2D Gaussians. Our new method presents three key innovations: (1) introducing tailored Gaussian kernels, (2) extending rasterization to X-ray imaging, and (3) developing a CUDA-based differentiable voxelizer. Experiments on synthetic and real-world datasets demonstrate that our method outperforms state-of-the-art approaches in accuracy and efficiency. Crucially, it delivers high-quality results in 4 minutes, which is 12$\times$ faster than NeRF-based methods and on par with traditional algorithms. Code and models are available on the project page https://github.com/Ruyi-Zha/r2_gaussian.

R$^2$-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction

TL;DR

R-Gaussian, the first 3DGS-based framework for sparse-view tomographic reconstruction, is introduced, which outperforms state-of-the-art approaches in accuracy and efficiency and delivers high-quality results in 4 minutes.

Abstract

3D Gaussian splatting (3DGS) has shown promising results in image rendering and surface reconstruction. However, its potential in volumetric reconstruction tasks, such as X-ray computed tomography, remains under-explored. This paper introduces R-Gaussian, the first 3DGS-based framework for sparse-view tomographic reconstruction. By carefully deriving X-ray rasterization functions, we discover a previously unknown integration bias in the standard 3DGS formulation, which hampers accurate volume retrieval. To address this issue, we propose a novel rectification technique via refactoring the projection from 3D to 2D Gaussians. Our new method presents three key innovations: (1) introducing tailored Gaussian kernels, (2) extending rasterization to X-ray imaging, and (3) developing a CUDA-based differentiable voxelizer. Experiments on synthetic and real-world datasets demonstrate that our method outperforms state-of-the-art approaches in accuracy and efficiency. Crucially, it delivers high-quality results in 4 minutes, which is 12 faster than NeRF-based methods and on par with traditional algorithms. Code and models are available on the project page https://github.com/Ruyi-Zha/r2_gaussian.
Paper Structure (49 sections, 11 equations, 18 figures, 4 tables)

This paper contains 49 sections, 11 equations, 18 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related work
  3. Tomographic reconstruction
  4. 3DGS
  5. Preliminary
  6. X-ray imaging
  7. 3D Gaussian splatting
  8. Method
  9. Representing objects with radiative Gaussians
  10. Initialization
  11. Training radiative Gaussians
  12. X-ray rasterization
  13. Transformation
  14. Projection and composition
  15. Integration bias
  16. ...and 34 more sections

Figures (18)

  • Figure 1: We compare our method to state-of-the-art NeRF-based methods (IntraTomo zang2021intratomo, NAF zha2022naf, SAX-NeRF cai2023structure) in terms of visual quality, PSNR (dB), and training time (minute). Our method achieves the highest reconstruction quality and is significantly faster than other methods.
  • Figure 2: A detection plane captures the attenuation of X-rays emitted from different angles.
  • Figure 3: We represent the scanned object as a set of radiative Gaussians. We optimize them using real X-ray projections and finally retrieve the density volume with voxelization.
  • Figure 4: Training pipeline of R$^2$-Gaussian. (a) Overall training pipeline. (b) X-ray rasterization for projection rendering. (c) Density voxelization for volume retrieval. (d) Modified adaptive control.
  • Figure 5: Density inconsistency in 3DGS.
  • ...and 13 more figures