Novel-view X-ray Projection Synthesis through Geometry-Integrated Deep Learning

TL;DR

This work tackles the need for novel-view X-ray projections without extensive multi-view acquisitions to reduce radiation exposure. It presents DL-GIPS, a geometry-integrated deep-learning framework that disentangles geometry and texture features, back-projects geometry into a 3D volume, forward-projects to target views, and uses a generator with a multi-scale discriminator to synthesize realistic X-ray projections. On the LIDC-IDRI lung dataset, DL-GIPS outperforms a UNet baseline in both one-to-one and multi-to-multi view synthesis across MAE, RMSE, SSIM, and PSNR (e.g., AP→LT MAE ≈ $0.052$, PSNR ≈ $19.46$; Multi→Multi MAE ≈ $0.017$, PSNR ≈ $23.53$), demonstrating improved accuracy and realism. The method holds promise for stereoscopic and volumetric X-ray imaging with lower data requirements, potentially benefiting pediatric and pregnant patients by reducing radiation dose and streamlining clinical workflows.

Abstract

X-ray imaging plays a crucial role in the medical field, providing essential insights into the internal anatomy of patients for diagnostics, image-guided procedures, and clinical decision-making. Traditional techniques often require multiple X-ray projections from various angles to obtain a comprehensive view, leading to increased radiation exposure and more complex clinical processes. This paper explores an innovative approach using the DL-GIPS model, which synthesizes X-ray projections from new viewpoints by leveraging a single existing projection. The model strategically manipulates geometry and texture features extracted from an initial projection to match new viewing angles. It then synthesizes the final projection by merging these modified geometry features with consistent texture information through an advanced image generation process. We demonstrate the effectiveness and broad applicability of the DL-GIPS framework through lung imaging examples, highlighting its potential to revolutionize stereoscopic and volumetric imaging by minimizing the need for extensive data acquisition.

Figures (3)

• Figure 1: Illustration of the DL-GIPS. The pipeline contains texture and geometry feature encoders, projection transformation and image generator.
• Figure 2: Illustration of one-to-one view synthesis experiments.
• Figure 3: Illustration of multi-to-multi view synthesis experiments.