AIA-UltraNeRF:Acoustic-Impedance-Aware Neural Radiance Field with Hash Encodings for Robotic Ultrasound Reconstruction and Localization

TL;DR

This work tackles the need for accurate 3D ultrasound reconstruction and robust localization by incorporating acoustic impedance into neural radiance field representations. It introduces AIA-UltraNeRF, which encodes impedance with multi-resolution hash grids and renders color through a directional MLP, enabling faster reconstruction and enabling offline initial pose estimation via hash-based image retrieval. A dual-supervised hashing framework with teacher–student models enhances localization robustness, while a spherical remote-center-of-motion (RCM) scanning mode decouples scanning from diagnostic workflows. Experimental results on phantoms and human subjects demonstrate improved reconstruction quality, faster inference (roughly 9.9× faster than vanilla NeRF), and accurate pose initialization for ultrasound localization, suggesting meaningful gains for operator-independent, CT/MRI-like workflow separation in clinical ultrasound.

Abstract

Neural radiance field (NeRF) is a promising approach for reconstruction and new view synthesis. However, previous NeRF-based reconstruction methods overlook the critical role of acoustic impedance in ultrasound imaging. Localization methods face challenges related to local minima due to the selection of initial poses. In this study, we design a robotic ultrasound system (RUSS) with an acoustic-impedance-aware ultrasound NeRF (AIA-UltraNeRF) to decouple the scanning and diagnostic processes. Specifically, AIA-UltraNeRF models a continuous function of hash-encoded spatial coordinates for the 3D ultrasound map, allowing for the storage of acoustic impedance without dense sampling. This approach accelerates both reconstruction and inference speeds. We then propose a dual-supervised network that leverages teacher and student models to hash-encode the rendered ultrasound images from the reconstructed map. AIA-UltraNeRF retrieves the most similar hash values without the need to render images again, providing an offline initial image position for localization. Moreover, we develop a RUSS with a spherical remote center of motion mechanism to hold the probe, implementing operator-independent scanning modes that separate image acquisition from diagnostic workflows. Experimental results on a phantom and human subjects demonstrate the effectiveness of acoustic impedance in implicitly characterizing the color of ultrasound images. AIAUltraNeRF achieves both reconstruction and localization with inference speeds that are 9.9 faster than those of vanilla NeRF.

Figures (16)

• Figure 1: The difference between camera and ultrasound imaging. (a) Camera imaging relies on dense samples of light rays to render color. (b) In ultrasound imaging, color is mapped by acoustic impedance at the corresponding position.
• Figure 2: (a) An overview of the AIA-UltraNeRF framework. The top panel shows the reconstruction module, where hash grids and spherical harmonics encode acoustic impedance and wave direction for NeRF-based ultrasound reconstruction, supervised by pixel-level loss. The bottom panel illustrates the localization module, where a teacher-student network learns hash codes of rendered images using multiple loss terms for ultrasound localization. (b) The encoding process of acoustic impedance by hash grids. (c) Ultrasound scans for the reconstruction module and the rendered new views with the standard plane.
• Figure 3: Ultrasonic waves are reflected by tissue with differing acoustic impedance to form ultrasound images, and ultrasound images are rendered by aggregating acoustic impedance features.
• Figure 4: Experimental setup: Simulation is conducted using the ABDFAN phantom, and the experiment involves human subjects.
• Figure 5: (a) Spherical series mechanism for ultrasound scanning. (b) Motion flexibility of the spherical RCM mechanism during scanning.