UltraG-Ray: Physics-Based Gaussian Ray Casting for Novel Ultrasound View Synthesis

Abstract

Novel view synthesis (NVS) in ultrasound has gained attention as a technique for generating anatomically plausible views beyond the acquired frames, offering new capabilities for training clinicians or data augmentation. However, current methods struggle with complex tissue and view-dependent acoustic effects. Physics-based NVS aims to address these limitations by including the ultrasound image formation process into the simulation. Recent approaches combine a learnable implicit scene representation with an ultrasound-specific rendering module, yet a substantial gap between simulation and reality remains. In this work, we introduce UltraG-Ray, a novel ultrasound scene representation based on a learnable 3D Gaussian field, coupled to an efficient physics-based module for B-mode synthesis. We explicitly encode ultrasound-specific parameters, such as attenuation and reflection, into a Gaussian-based spatial representation and realize image synthesis within a novel ray casting scheme. In contrast to previous methods, this approach naturally captures view-dependent attenuation effects, thereby enabling the generation of physically informed B-mode images with increased realism. We compare our method to state-of-the-art and observe consistent gains in image quality metrics (up to 15% increase on MS-SSIM), demonstrating clear improvement in terms of realism of the synthesized ultrasound images.

Figures (7)

• Figure 1: Overview of UltraG-Ray pipeline: a) Progressive adaptation of the learnable 3D Gaussian distribution to match the acquired data b) Image synthesis pipeline: First Gaussians are filtered based on the respective pose, second Gaussian ray-intersection is used to create the echo and transmittance maps and finally the resulting B-Mode image c) Downstream task evaluation based on NVS, where green and orange frames denote novel views
• Figure 2: Depiction of the attenuation and intensity formation process: a) Transmittance accumulation, where each Gaussian gradually reduces the remaining energy along the ray. b) Computation of pixel-wise intensity contributions from view-dependent Gaussian intensity and their distance-weighted influence. c) Final pixel intensity obtained by combining accumulated attenuation with the weighted Gaussian contributions.
• Figure 3: Qualitative comparison of UltraG-Ray and baselines on the spine phantom.
• Figure 4: NVS comparison on ex vivo porcine muscle. Highlighted regions (red and orange) are enlarged as cut-outs on the right side of the respective image.
• Figure 5: NVS examples from both datasets for the intermediate echo and transmittance maps, the synthesized B-mode output, and the corresponding ground truth image.