Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Coarse-Fine View Attention Alignment-Based GAN for CT Reconstruction from Biplanar X-Rays

Zhi Qiao, Hanqiang Ouyang, Dongheng Chu, Huishu Yuan, Xiantong Zhen, Pei Dong, Zhen Qian

TL;DR

This work tackles 3D CT reconstruction from biplanar X-rays to aid surgical planning by exploiting complementary information from orthogonal views. The authors propose CVAA-GAN, a generative framework that uses a two-branch encoder with a CVAA (coarse-fine view attention alignment) module in the decoder to fuse view-specific features, augmented by a fine-distillation path and a Patch-3D discriminator. The training combines an adversarial loss under a least-squares GAN objective and a projection-based reconstruction loss on three orthogonal planes to enforce global shape consistency. On a lumbar vertebra DRR-based dataset, CVAA-GAN outperforms PSR, 3DCNN, and X2CT-GAN across multiple metrics (e.g., MAE, MSE, Cosine Similarity, PSNR, SSIM), demonstrating improved preservation of anatomy and sharper boundaries with potential clinical impact for intraoperative imaging when CT is unavailable.

Abstract

For surgical planning and intra-operation imaging, CT reconstruction using X-ray images can potentially be an important alternative when CT imaging is not available or not feasible. In this paper, we aim to use biplanar X-rays to reconstruct a 3D CT image, because biplanar X-rays convey richer information than single-view X-rays and are more commonly used by surgeons. Different from previous studies in which the two X-ray views were treated indifferently when fusing the cross-view data, we propose a novel attention-informed coarse-to-fine cross-view fusion method to combine the features extracted from the orthogonal biplanar views. This method consists of a view attention alignment sub-module and a fine-distillation sub-module that are designed to work together to highlight the unique or complementary information from each of the views. Experiments have demonstrated the superiority of our proposed method over the SOTA methods.

Coarse-Fine View Attention Alignment-Based GAN for CT Reconstruction from Biplanar X-Rays

TL;DR

This work tackles 3D CT reconstruction from biplanar X-rays to aid surgical planning by exploiting complementary information from orthogonal views. The authors propose CVAA-GAN, a generative framework that uses a two-branch encoder with a CVAA (coarse-fine view attention alignment) module in the decoder to fuse view-specific features, augmented by a fine-distillation path and a Patch-3D discriminator. The training combines an adversarial loss under a least-squares GAN objective and a projection-based reconstruction loss on three orthogonal planes to enforce global shape consistency. On a lumbar vertebra DRR-based dataset, CVAA-GAN outperforms PSR, 3DCNN, and X2CT-GAN across multiple metrics (e.g., MAE, MSE, Cosine Similarity, PSNR, SSIM), demonstrating improved preservation of anatomy and sharper boundaries with potential clinical impact for intraoperative imaging when CT is unavailable.

Abstract

For surgical planning and intra-operation imaging, CT reconstruction using X-ray images can potentially be an important alternative when CT imaging is not available or not feasible. In this paper, we aim to use biplanar X-rays to reconstruct a 3D CT image, because biplanar X-rays convey richer information than single-view X-rays and are more commonly used by surgeons. Different from previous studies in which the two X-ray views were treated indifferently when fusing the cross-view data, we propose a novel attention-informed coarse-to-fine cross-view fusion method to combine the features extracted from the orthogonal biplanar views. This method consists of a view attention alignment sub-module and a fine-distillation sub-module that are designed to work together to highlight the unique or complementary information from each of the views. Experiments have demonstrated the superiority of our proposed method over the SOTA methods.
Paper Structure (14 sections, 1 equation, 2 figures, 1 table)

This paper contains 14 sections, 1 equation, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. CVAA-GAN
  3. Generator
  4. Encoder
  5. Decoder
  6. Objective Functions
  7. Adversarial Loss
  8. Reconstruction Loss
  9. Experiments
  10. Datasets
  11. Settings
  12. Qualitative Results
  13. Quantitative Results
  14. Conclusions

Figures (2)

  • Figure 1: Network architecture of the CVAA-GAN Generator
  • Figure 2: Representative CT reconstruction results shown in the mid-axial (1st row), mid-sagittal (2nd row), and mid-coronal (3rd row) views. Our method is compared with baseline methods (PSR, 3DCNN, X2CT-GAN) and ground truth (GT).