Training-free image style alignment for self-adapting domain shift on handheld ultrasound devices

TL;DR

The proposed TISA can directly infer handheld device images without extra training and is suited for clinical applications and shows that TISA performs better and more stably in medical detection and segmentation tasks for handheld device data.

Abstract

Handheld ultrasound devices face usage limitations due to user inexperience and cannot benefit from supervised deep learning without extensive expert annotations. Moreover, the models trained on standard ultrasound device data are constrained by training data distribution and perform poorly when directly applied to handheld device data. In this study, we propose the Training-free Image Style Alignment (TISA) framework to align the style of handheld device data to those of standard devices. The proposed TISA can directly infer handheld device images without extra training and is suited for clinical applications. We show that TISA performs better and more stably in medical detection and segmentation tasks for handheld device data. We further validate TISA as the clinical model for automatic measurements of spinal curvature and carotid intima-media thickness. The automatic measurements agree well with manual measurements made by human experts and the measurement errors remain within clinically acceptable ranges. We demonstrate the potential for TISA to facilitate automatic diagnosis on handheld ultrasound devices and expedite their eventual widespread use.

Figures (4)

• Figure 1: Training-free image style alignment (TISA) for ultrasound image analysis on handheld devices. We trained models on source data and directly applied them to target data through TISA. The diffusion model aligned the target image style to resemble the source data while ensuring structural integrity. The uncertainty-aware model predicted all aligned images and selected the best one by using uncertainty. We collected source data from three standard devices and target data from two handheld devices through spinal and carotid examinations. We conducted the evaluations on detection, segmentation, and automatic measurements.
• Figure 2: Comparison of domain adaptation methods for vertebral structure detection on the handheld device data.a, The accuracy, recall, F1 score, and mean error of our method compared to other adaptation methods when tested on the target data (n=1469). Each result was an average of five trials with different random initializations and error bars depicting the standard error of the mean (SEM). b, UMAP plots illustrating source, target, and aligned target data in the feature space. TISA effectively eliminated domain shifts between source and target data. c, Examples of target data and the corresponding aligned images. TISA aligned images with the source-like style while preserving structural details.
• Figure 3: Assessment of TISA in automatic spinal curvature measurement on clinical data collected from the handheld device. a, Analysis between ultrasound manual and automatic measurements (106 subjects, 79,790 transverse slices, n = 165). The manual measurements are acquired onsite by skilled experts. b, Analysis between ultrasound automatic and radiographic manual measurements (49 subjects, n = 73). The radiographic measurements have been implemented at various other hospitals as the gold standard diagnostic results. We exclude cases that are too outdated or involve subjects who have received treatment. c, Visualization of two measurement methods on mild ($10^{\circ}\sim25^{\circ}$) and moderate ( $25^{\circ}\sim40^{\circ}$) spinal curvatures.
• Figure 4: Performance of TISA in carotid intima-media segmentation and thickness measurement using handheld device data. a, Boxplot comparison of Dice score and IoU between all adaptation methods over multiple runs. b, Correlation and Bland–Altman analysis between manual and automatic measurement (n = 76). c, Aligned images with their segmentation results and automatic thickness measurements. The thickness plotted on the target image was manually measured by tracing the contours of the lumen-intima (LI, red line) and media-adventitia (MA, green line) anatomical interfaces.