UlRe-NeRF: 3D Ultrasound Imaging through Neural Rendering with Ultrasound Reflection Direction Parameterization

TL;DR

UlRe-NeRF addresses the challenge of high-fidelity 3D ultrasound imaging by integrating implicit neural representations with physics-informed ultrasound volume rendering. It introduces Ultrasound Reflection Direction Parameterization and Reflective Harmonic Encoding to capture high-frequency, view-dependent reflections, while a spatial MLP predicts tissue properties that drive volume rendering. The rendering pipeline combines reflection intensity with Beer-Lambert-inspired attenuation, a 2D PSF, and a final fusion step to produce realistic ultrasound images. On a liver ultrasound dataset, UlRe-NeRF outperforms NeRF-based baselines and Ultra-NeRF in perceptual quality metrics, validating a physics-guided neural rendering approach and highlighting potential clinical impact, with acknowledged future needs in sample efficiency and handling sparse data from freehand scanning.

Abstract

Three-dimensional ultrasound imaging is a critical technology widely used in medical diagnostics. However, traditional 3D ultrasound imaging methods have limitations such as fixed resolution, low storage efficiency, and insufficient contextual connectivity, leading to poor performance in handling complex artifacts and reflection characteristics. Recently, techniques based on NeRF (Neural Radiance Fields) have made significant progress in view synthesis and 3D reconstruction, but there remains a research gap in high-quality ultrasound imaging. To address these issues, we propose a new model, UlRe-NeRF, which combines implicit neural networks and explicit ultrasound volume rendering into an ultrasound neural rendering architecture. This model incorporates reflection direction parameterization and harmonic encoding, using a directional MLP module to generate view-dependent high-frequency reflection intensity estimates, and a spatial MLP module to produce the medium's physical property parameters. These parameters are used in the volume rendering process to accurately reproduce the propagation and reflection behavior of ultrasound waves in the medium. Experimental results demonstrate that the UlRe-NeRF model significantly enhances the realism and accuracy of high-fidelity ultrasound image reconstruction, especially in handling complex medium structures.

Figures (6)

• Figure 1: UlRe-NeRF for High-Fidelity 3D Ultrasound Imaging. This figure illustrates our proposed ultrasound neural rendering architecture, which integrates implicit neural representation (b) and volume rendering (c) to enhance 3D ultrasound imaging. The system comprises two main modules: the spatial MLP and the directional MLP. The spatial MLP predicts physical property parameters, such as attenuation coefficients, reflectivity, scattering density, and amplitude, through high-dimensional position encoding. This simulates the propagation and reflection behavior of ultrasound waves within the medium. The directional MLP processes the input ray directions, generating view-dependent intensity estimates that capture complex reflection phenomena.
• Figure 2: Ultrasound Wave Interaction and Reflection Parameterization at the Medium Interface. (a) Shows the reflection and refraction phenomena when the ultrasound wave $\boldsymbol{I}_i$ impinges on the medium interface, describing the basic physical processes and corresponding formulas. (b) Describes the process of calculating the specular reflection direction $\hat{\boldsymbol{I}}_{r s}$ by adjusting the macroscale normal vector $\widehat{\mathbf{n}}$ to the microfacet normal vector $\widehat{\mathbf{n}}^{\prime}$, highlighting the impact of the microfacet structure on reflection characteristics.
• Figure 3: Ultrasound Neural Rendering Architecture of UlRe-NeRF. This figure illustrates our proposed ultrasound neural rendering architecture. The view-dependent reflection intensity information and physical property parameters generated by the directional MLP and spatial MLP modules are combined in the rendering section, achieving high-fidelity simulation of ultrasound wave propagation and reflection behavior.
• Figure 4: Our model demonstrates accurate and high-fidelity performance, which highlights two different frames of ultrasound images. (a) Overall image comparison; (b) Detailed section comparison.
• Figure 5: Visual Comparison of Different Rendering Methods. Ours Orig. indicates the use of Ultra-NeRF's rendering method, while Ultra-NeRF New indicates the use of our rendering method.