Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SynStitch: a Self-Supervised Learning Network for Ultrasound Image Stitching Using Synthetic Training Pairs and Indirect Supervision

Xing Yao, Runxuan Yu, Dewei Hu, Hao Yang, Ange Lou, Jiacheng Wang, Daiwei Lu, Gabriel Arenas, Baris Oguz, Alison Pouch, Nadav Schwartz, Brett C Byram, Ipek Oguz

TL;DR

This work introduces SynStitch, a self-supervised framework designed for 2DUS stitching that was evaluated against multiple leading methods on a kidney ultrasound dataset, demonstrating superior 2DUS stitching performance through both qualitative and quantitative analyses.

Abstract

Ultrasound (US) image stitching can expand the field-of-view (FOV) by combining multiple US images from varied probe positions. However, registering US images with only partially overlapping anatomical contents is a challenging task. In this work, we introduce SynStitch, a self-supervised framework designed for 2DUS stitching. SynStitch consists of a synthetic stitching pair generation module (SSPGM) and an image stitching module (ISM). SSPGM utilizes a patch-conditioned ControlNet to generate realistic 2DUS stitching pairs with known affine matrix from a single input image. ISM then utilizes this synthetic paired data to learn 2DUS stitching in a supervised manner. Our framework was evaluated against multiple leading methods on a kidney ultrasound dataset, demonstrating superior 2DUS stitching performance through both qualitative and quantitative analyses. The code will be made public upon acceptance of the paper.

SynStitch: a Self-Supervised Learning Network for Ultrasound Image Stitching Using Synthetic Training Pairs and Indirect Supervision

TL;DR

This work introduces SynStitch, a self-supervised framework designed for 2DUS stitching that was evaluated against multiple leading methods on a kidney ultrasound dataset, demonstrating superior 2DUS stitching performance through both qualitative and quantitative analyses.

Abstract

Ultrasound (US) image stitching can expand the field-of-view (FOV) by combining multiple US images from varied probe positions. However, registering US images with only partially overlapping anatomical contents is a challenging task. In this work, we introduce SynStitch, a self-supervised framework designed for 2DUS stitching. SynStitch consists of a synthetic stitching pair generation module (SSPGM) and an image stitching module (ISM). SSPGM utilizes a patch-conditioned ControlNet to generate realistic 2DUS stitching pairs with known affine matrix from a single input image. ISM then utilizes this synthetic paired data to learn 2DUS stitching in a supervised manner. Our framework was evaluated against multiple leading methods on a kidney ultrasound dataset, demonstrating superior 2DUS stitching performance through both qualitative and quantitative analyses. The code will be made public upon acceptance of the paper.

Paper Structure

This paper contains 7 sections, 2 equations, 3 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methods
  3. Synthetic Stitching Pair Generation Module (SSPGM)
  4. Image Stitching Module (ISM)
  5. Experimental Design
  6. Results
  7. Conclusion

Figures (3)

  • Figure 1: SynStitch overview. We first train the SSPGM to generate a realistic 2DUS image $I_s$ from an input image $I$ with a random affine matrix $\mathcal{A}$. Then we freeze the SSPGM and we train ISM on the synthetic stitching pairs.
  • Figure 2: Overview of the SSPGM. (a) Training: The SSPGM is trained to learn outpainting given the outpainting condition $C$. (b) Inference: The trained SSPGM performs outpainting under the specified condition $C$. (c) Stitching pair generation: For a single input image $I$, a synthetic condition $C_{s}$ is generated and fed into the pre-trained SSPGM, which then generates $I_{s}$ as a stitching pair for $I$, with associated affine matrix $\mathcal{A}$. (d) SSPGM results with custom affine matrices. T, R, S indicate translation, rotation, scaling. SSPGM can generate synthetic 2DUS images with a sequence of affine transformations.
  • Figure 3: Stitching results from randomly selected samples from two RealStitch subjects. Yellow contours: registered source images; blue contours: fixed images. Blue tags: source and target images; gray tags: conventional methods; purple tags: DL methods; yellow tags: the two variants of our proposed model, SynStitch-VXM (SS-VXM) and SynStitch-GLN (SS-GLN); red tag: ground truth. ANTs and DL methods only align the FOV and fail to align anatomy. SIFT attempts to align anatomy but is not accurate. Our method produces robust results.