Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

DDGS-CT: Direction-Disentangled Gaussian Splatting for Realistic Volume Rendering

Zhongpai Gao, Benjamin Planche, Meng Zheng, Xiao Chen, Terrence Chen, Ziyan Wu

TL;DR

DDGS presents a novel, differentiable DRR renderer that blends physics-informed X-ray interactions with efficient 3D Gaussian splatting. By disentangling isotropic and direction-dependent contributions and using Radiodensity-Aware Dual Sampling for initialization, it achieves higher image fidelity than prior 3DGS and X-Gaussian approaches while using far fewer primitives. The method yields improved novel-view DRR quality and faster, more accurate 2D/3D CT pose registrations, highlighting its potential for real-time intraoperative use. Limitations include reliance on offline ground-truth renderers for training and potential breakdown in highly complex multi-bounce scattering, pointing to future extensions with higher-order splatting and noise compensation. Overall, DDGS offers a practical, scalable path toward realistic and differentiable DRRs suitable for intraoperative tasks and inverse problems.

Abstract

Digitally reconstructed radiographs (DRRs) are simulated 2D X-ray images generated from 3D CT volumes, widely used in preoperative settings but limited in intraoperative applications due to computational bottlenecks, especially for accurate but heavy physics-based Monte Carlo methods. While analytical DRR renderers offer greater efficiency, they overlook anisotropic X-ray image formation phenomena, such as Compton scattering. We present a novel approach that marries realistic physics-inspired X-ray simulation with efficient, differentiable DRR generation using 3D Gaussian splatting (3DGS). Our direction-disentangled 3DGS (DDGS) method separates the radiosity contribution into isotropic and direction-dependent components, approximating complex anisotropic interactions without intricate runtime simulations. Additionally, we adapt the 3DGS initialization to account for tomography data properties, enhancing accuracy and efficiency. Our method outperforms state-of-the-art techniques in image accuracy. Furthermore, our DDGS shows promise for intraoperative applications and inverse problems such as pose registration, delivering superior registration accuracy and runtime performance compared to analytical DRR methods.

DDGS-CT: Direction-Disentangled Gaussian Splatting for Realistic Volume Rendering

TL;DR

DDGS presents a novel, differentiable DRR renderer that blends physics-informed X-ray interactions with efficient 3D Gaussian splatting. By disentangling isotropic and direction-dependent contributions and using Radiodensity-Aware Dual Sampling for initialization, it achieves higher image fidelity than prior 3DGS and X-Gaussian approaches while using far fewer primitives. The method yields improved novel-view DRR quality and faster, more accurate 2D/3D CT pose registrations, highlighting its potential for real-time intraoperative use. Limitations include reliance on offline ground-truth renderers for training and potential breakdown in highly complex multi-bounce scattering, pointing to future extensions with higher-order splatting and noise compensation. Overall, DDGS offers a practical, scalable path toward realistic and differentiable DRRs suitable for intraoperative tasks and inverse problems.

Abstract

Digitally reconstructed radiographs (DRRs) are simulated 2D X-ray images generated from 3D CT volumes, widely used in preoperative settings but limited in intraoperative applications due to computational bottlenecks, especially for accurate but heavy physics-based Monte Carlo methods. While analytical DRR renderers offer greater efficiency, they overlook anisotropic X-ray image formation phenomena, such as Compton scattering. We present a novel approach that marries realistic physics-inspired X-ray simulation with efficient, differentiable DRR generation using 3D Gaussian splatting (3DGS). Our direction-disentangled 3DGS (DDGS) method separates the radiosity contribution into isotropic and direction-dependent components, approximating complex anisotropic interactions without intricate runtime simulations. Additionally, we adapt the 3DGS initialization to account for tomography data properties, enhancing accuracy and efficiency. Our method outperforms state-of-the-art techniques in image accuracy. Furthermore, our DDGS shows promise for intraoperative applications and inverse problems such as pose registration, delivering superior registration accuracy and runtime performance compared to analytical DRR methods.
Paper Structure (28 sections, 5 equations, 7 figures, 9 tables)

This paper contains 28 sections, 5 equations, 7 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Motivation.
  3. Contributions.
  4. Related Work
  5. X-ray Image Simulation.
  6. Efficient and Differentiable DRR Generation.
  7. Methodology
  8. Preliminaries
  9. Gaussian Splatting.
  10. Application to DRR.
  11. Disentanglement of Isotropic and Anisotropic 3D Gaussians
  12. Initialization via Radiodensity-Aware Dual Sampling (RADS)
  13. Experiments
  14. Experimental Protocol
  15. Implementation.
  16. ...and 13 more sections

Figures (7)

  • Figure 1: Proposed pipeline of Direction-Disentangled 3D Gaussian Splatting (DDGS).
  • Figure 2: Illustration of the different sampling strategies for 3DGS initialization.
  • Figure 3: Visualization of the Gaussian cloud optimization for different methods.
  • Figure 4: Qualitative comparison of DRRs and real scans from DeepFluoro and Ljubljana datasets.
  • Figure 5: Visualization of the isotropic and anisotropic X-ray contributions to the rendered DRRs.
  • ...and 2 more figures