Generative neural physics enables quantitative volumetric ultrasound of tissue mechanics

TL;DR

The paper tackles quantitative tissue biomechanics imaging with ultrasound tomography by bridging physics-based wave propagation with generative AI. It introduces Real2Sim2Real: a Reality-to-Simulation pipeline that creates anatomically and physically realistic UT phantoms and a Strong Scattering Neural Operator (S^2NO) that serves as a physics-informed neural PDE solver for rapid, high-fidelity 3D full-waveform inversion. Trained solely on synthetic data, S^2NO delivers MRI-comparable 3D reconstructions of breast and musculoskeletal tissues in under ten minutes and provides quantitative maps of mechanical properties, including bone and muscle stiffness. The approach demonstrates MRI-grade resolution across breast, arm, and leg imaging with substantial speedups and robust generalization to different hardware setups, signaling potential for widespread clinical adoption of UT biomechanical imaging.

Abstract

Tissue mechanics--stiffness, density and impedance contrast--are broadly informative biomarkers across diseases, yet routine CT, MRI, and B-mode ultrasound rarely quantify them directly. While ultrasound tomography (UT) is intrinsically suited to in-vivo biomechanical assessment by capturing transmitted and reflected wavefields, efficient and accurate full-wave scattering models remain a bottleneck. Here, we introduce a generative neural physics framework that fuses generative models with physics-informed partial differential equation (PDE) solvers to produce rapid, high-fidelity 3D quantitative imaging of tissue mechanics. A compact neural surrogate for full-wave propagation is trained on limited cross-modality data, preserving physical accuracy while enabling efficient inversion. This enables, for the first time, accurate and efficient quantitative volumetric imaging of in vivo human breast and musculoskeletal tissues in under ten minutes, providing spatial maps of tissue mechanical properties not available from conventional reflection-mode or standard UT reconstructions. The resulting images reveal biomechanical features in bone, muscle, fat, and glandular tissues, maintaining structural resolution comparable to 3T MRI while providing substantially greater sensitivity to disease-related tissue mechanics.

Figures (11)

• Figure 1: Schematic illustration of the proposed generative neural physics framework. (a) An annular transducer array equipped with motorized retractable bellows is employed to acquire slice-wise ultrasound tomography (UT) data. (b) A pretrained generative model, fine-tuned with limited cross-modality data, generates a large ensemble of anatomically realistic acoustic property maps across multiple organs. A high-fidelity numerical solver simulates the corresponding acoustic wavefields under realistic experimental conditions, producing paired datasets to train the Strong Scattering Neural Operator ($S^2NO$) as an efficient digital instrument surrogate. During UT imaging, $S^2NO$ acts as a forward surrogate model to rapidly reconstruct tissue mechanical properties from measured wavefield data. (c) Time-domain signals obtained from a single excitation contain scattered components arising from different anatomical structures (e.g. skin, bone). (d) After preprocessing, the time-domain signals are converted into multiple discrete frequency-domain data for UT image reconstruction. (e) Transverse sections from the 3D reconstruction of a female leg obtained using the generative neural physics framework, showing distinct mechanical contrasts among different anatomical structures. BF, biceps femoris; GSV, great saphenous vein.
• Figure 2: Generation of anatomically realistic tissue sound speed phantoms. (a) Clinical images from alternative imaging modalities (e.g., CT) are segmented using the Segment Anything Model (SAM) and subsequently converted into sound-speed maps based on established acoustic properties of biological tissues. A base Stable Diffusion model generates phantoms with semantically coherent morphology but limited anatomical realism. After fine-tuning with an anatomically realistic tissue phantom library, the model yields a large set of morphologically faithful tissue sound-speed samples. (b) Representative sound-speed maps of breast, arm, and leg tissues generated by the model and the corresponding wavefields, both closely resembling in vivo samples.
• Figure 3: The architecture and forward simulation result of $S^2NO$. (a) Schematic overview of the entire $S^2NO$ architecture. The sound speed and incident field are encoded into a latent space as three auxiliary variables representing the wavefield (u), scattering potential (v) and preconditioner (q). Within the latent space, the wavefield is iteratively updated using a learnable scattering mechanism, where each iteration corresponds to an $S^2NO$ layer. A decoder then projects the latent variables to the physical space to produce the predicted wavefield. (b) Wavefield predictions from $S^2NO$ and baseline models for breast, arm, and leg phantoms at 0.6MHz. $S^2NO$ accurately captures complex scattering patterns, particularly at the marrow cavity and the skin-air interface. (c) Forward simulation errors of $S^2NO$ and baseline models for breast, arm, and leg phantoms at a 0.6MHz setting. Boxplots show the median (central line) and interquartile range (IQR, box); whiskers extend to the smallest and largest values within 1.5× the IQR (outliers omitted), sample size N=320/373/108 for Breast/Arm/Leg dataset. (d) Comparison of forward simulation errors between $S^2NO$ and the baseline models for all organ phantoms at frequencies of 0.25 0.6 MHz. The setting of the boxplots is the same as (c), sample size N=801. Despite the increased challenge at higher frequencies, $S^2NO$ consistently outperforms all baseline methods at every frequency and for every organ.
• Figure 4: Quantitative ultrasound tomography pipeline and reconstruction performance of$S^2NO$ on synthetic tissue phantoms. (a)The $S^2NO$ is deployed as a forward surrogate in UT imaging to solve a full waveform inversion problem. It generates scattered wavefields from the estimated sound speed model, and the residuals between simulated and measured data at sensor positions are backpropagated to iteratively update the model via adjoint method. (b) Comparison between the reference breast, arm and leg phantoms and the reconstructed phantoms using different model (numerical solver, $S^2NO$, FNO and UNet). (c) Statistical summary of SSIM values for reconstruction results of breast, arm, and leg phantom in test dataset using various models (numerical solver, $S^2NO$, FNO and UNet). Boxplots show the median (central line) and interquartile range (IQR, box); whiskers extend to the smallest and largest values within 1.5× the IQR (outliers omitted), sample size N=24/16/12 for Breast/Arm/Leg dataset. (d) Distribution of SSIM differences between $S^2NO$ and other methods computed on a sample-by-sample basis. $S^2NO$ matches the imaging quality of conventional numerical solver while outperforming all other baseline approaches.
• Figure 5: The UT imaging of two breasts with malignancy or benign cyst. (a-b) Comparison of reconstructions of breasts with a spiculated malignancy (a) and a benign cyst (b), obtained using various imaging techniques (FWI, $S^2NO$-FWI, ToFT, and DAS). $S^2NO$-FWI provides image quality comparable to traditional FWI solvers, particularly in resolving critical structures such as tumors and glandular tissue, whereas DAS and ToFT fail to reconstruct meaningful internal anatomy. (c) Reconstruction accuracy of $S^2NO$-FWI improves progressively with increasing dataset bandwidth. (d) Schematic illustration of cross-sectional data acquisition for breast imaging. (e-f) One-dimensional velocity profiles extracted at specific positions (red line) from the reconstructed breast model with a spiculated malignancy (e) and a benign cyst (f). (g) Computational time per iteration for traditional solver-based FWI and $S^2NO$-FWI across varying frequencies. $S^2NO$ accelerates computations by over tenfold. The data points denote mean values, and the vertical bars indicate the standard deviation, sample size N=10.