Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

UMedNeRF: Uncertainty-aware Single View Volumetric Rendering for Medical Neural Radiance Fields

Jing Hu, Qinrui Fan, Shu Hu, Siwei Lyu, Xi Wu, Xin Wang

TL;DR

This work tackles the high radiation burden of CT by reconstructing CT-like projections from a single X-ray using UMedNeRF, an uncertainty-aware radiance-field model. The method leverages an uncertainty-driven multitask objective to automatically balance perceptual (L_r) and reconstruction (L_{MSE}) losses during training, enabling a single-view input to generate consistent 3D CT projections across a full rotational view. Key contributions include automatic weighting of loss terms via task uncertainty, a GAN-augmented NeRF framework for medical imaging, and demonstrated improvements in PSNR/SSIM over baselines with evidence of enhanced perceptual quality (FID/KID) on chest and knee DRRs. This approach holds potential for dose reduction and faster CT-like visualization in clinical settings, with implications for bone trauma and surgical planning, though it does not yet replace conventional CT imaging.

Abstract

In the field of clinical medicine, computed tomography (CT) is an effective medical imaging modality for the diagnosis of various pathologies. Compared with X-ray images, CT images can provide more information, including multi-planar slices and three-dimensional structures for clinical diagnosis. However, CT imaging requires patients to be exposed to large doses of ionizing radiation for a long time, which may cause irreversible physical harm. In this paper, we propose an Uncertainty-aware MedNeRF (UMedNeRF) network based on generated radiation fields. The network can learn a continuous representation of CT projections from 2D X-ray images by obtaining the internal structure and depth information and using adaptive loss weights to ensure the quality of the generated images. Our model is trained on publicly available knee and chest datasets, and we show the results of CT projection rendering with a single X-ray and compare our method with other methods based on generated radiation fields.

UMedNeRF: Uncertainty-aware Single View Volumetric Rendering for Medical Neural Radiance Fields

TL;DR

This work tackles the high radiation burden of CT by reconstructing CT-like projections from a single X-ray using UMedNeRF, an uncertainty-aware radiance-field model. The method leverages an uncertainty-driven multitask objective to automatically balance perceptual (L_r) and reconstruction (L_{MSE}) losses during training, enabling a single-view input to generate consistent 3D CT projections across a full rotational view. Key contributions include automatic weighting of loss terms via task uncertainty, a GAN-augmented NeRF framework for medical imaging, and demonstrated improvements in PSNR/SSIM over baselines with evidence of enhanced perceptual quality (FID/KID) on chest and knee DRRs. This approach holds potential for dose reduction and faster CT-like visualization in clinical settings, with implications for bone trauma and surgical planning, though it does not yet replace conventional CT imaging.

Abstract

In the field of clinical medicine, computed tomography (CT) is an effective medical imaging modality for the diagnosis of various pathologies. Compared with X-ray images, CT images can provide more information, including multi-planar slices and three-dimensional structures for clinical diagnosis. However, CT imaging requires patients to be exposed to large doses of ionizing radiation for a long time, which may cause irreversible physical harm. In this paper, we propose an Uncertainty-aware MedNeRF (UMedNeRF) network based on generated radiation fields. The network can learn a continuous representation of CT projections from 2D X-ray images by obtaining the internal structure and depth information and using adaptive loss weights to ensure the quality of the generated images. Our model is trained on publicly available knee and chest datasets, and we show the results of CT projection rendering with a single X-ray and compare our method with other methods based on generated radiation fields.
Paper Structure (6 sections, 5 equations, 3 figures, 3 tables)

This paper contains 6 sections, 5 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. METHODOLOGY
  3. Experiments
  4. Conclusion
  5. COMPLIANCE WITH ETHICAL STANDARDS
  6. Acknowledgments

Figures (3)

  • Figure 1: Overview of the task of UMedNeRF. Top: The goal is to render the 3D volume from a single view. Bottom: Given the trained Generator, it is further fine-tuned for the final rendering process via a multitask learning method.
  • Figure 2: Overview of our Uncertainty-aware MedNeRF. Left: Network structures. Right: (Top) Fine-tuning the generator for balancing the Blurrinenss and Accuracy. (Bottom) Rendering the whole 3D volume (each view taken at a 5-degree interval) using a single view slide.
  • Figure 3: The single knee X-ray given yields a complete CT projection by our uncertainty-aware fine-tuned generator.