Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

FermatSyn: SAM2-Enhanced Bidirectional Mamba with Isotropic Spiral Scanning for Multi-Modal Medical Image Synthesis

Feng Yuan

TL;DR

Experiments show FermatSyn surpasses state-of-the-art methods in PSNR, SSIM, FID, and 3D structural consistency, and downstream segmentation on synthesized images yields no significant difference from real-image training, confirming clinical utility.

Abstract

Multi-modal medical image synthesis is pivotal for alleviating clinical data scarcity, yet existing methods fail to reconcile global anatomical consistency with high-fidelity local detail. We propose FermatSyn, which addresses three persistent limitations: (1)~a SAM2-based Prior Encoder that injects domain-aware anatomical knowledge via Lo-RA$^{+}$ efficient fine-tuning of a frozen SAM2 Vision Transformer; (2)~a Hierarchical Residual Downsampling Module (HRDM) coupled with a Cross-scale Integration Network (CIN) that preserves high-frequency lesion details and adaptively fuses global--local representations; and (3)~a continuity constrained Fermat Spiral Scanning strategy within a Bidirectional Fermat Scan Mamba (BFS-Mamba), constructing an approximately isotropic receptive field that substantially reduces the directional bias of raster or spiral serialization. Experiments on SynthRAD2023, BraTS2019, BraTS-MEN, and BraTS-MET show FermatSyn surpasses state-of-the-art methods in PSNR, SSIM, FID, and 3D structural consistency. Downstream segmentation on synthesized images yields no significant difference from real-image training ($p{>}0.05$), confirming clinical utility. Code will be released upon publication. \keywords{Medical image synthesis \and SAM2 \and Mamba \and Fermat spiral scanning \and Anatomical prior \and Cross-modal}

FermatSyn: SAM2-Enhanced Bidirectional Mamba with Isotropic Spiral Scanning for Multi-Modal Medical Image Synthesis

TL;DR

Experiments show FermatSyn surpasses state-of-the-art methods in PSNR, SSIM, FID, and 3D structural consistency, and downstream segmentation on synthesized images yields no significant difference from real-image training, confirming clinical utility.

Abstract

Multi-modal medical image synthesis is pivotal for alleviating clinical data scarcity, yet existing methods fail to reconcile global anatomical consistency with high-fidelity local detail. We propose FermatSyn, which addresses three persistent limitations: (1)~a SAM2-based Prior Encoder that injects domain-aware anatomical knowledge via Lo-RA efficient fine-tuning of a frozen SAM2 Vision Transformer; (2)~a Hierarchical Residual Downsampling Module (HRDM) coupled with a Cross-scale Integration Network (CIN) that preserves high-frequency lesion details and adaptively fuses global--local representations; and (3)~a continuity constrained Fermat Spiral Scanning strategy within a Bidirectional Fermat Scan Mamba (BFS-Mamba), constructing an approximately isotropic receptive field that substantially reduces the directional bias of raster or spiral serialization. Experiments on SynthRAD2023, BraTS2019, BraTS-MEN, and BraTS-MET show FermatSyn surpasses state-of-the-art methods in PSNR, SSIM, FID, and 3D structural consistency. Downstream segmentation on synthesized images yields no significant difference from real-image training (), confirming clinical utility. Code will be released upon publication. \keywords{Medical image synthesis \and SAM2 \and Mamba \and Fermat spiral scanning \and Anatomical prior \and Cross-modal}
Paper Structure (11 sections, 10 equations, 3 figures, 5 tables)

This paper contains 11 sections, 10 equations, 3 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Method
  3. Hybrid Encoder
  4. Bidirectional Fermat-Scan Mamba (BFS-Mamba)
  5. Training Objective
  6. Experiments
  7. Setup
  8. Quantitative Results
  9. Downstream Clinical Validation
  10. Ablation Studies
  11. Conclusion

Figures (3)

  • Figure 1: Overall architecture of FermatSyn. The Hybrid Encoder fuses SAM2 global priors with HRDM local details via CIN. BFS-Mamba processes the Fermat-serialised features bidirectionally and reconstructs the target modality through a residual decoder.
  • Figure 2: Scanning strategies and their empirically measured SSM operator footprints (Jacobian-based sensitivity maps). (a) Raster scan: pronounced horizontal-stripe directional bias ($\mu{=}0.891$, $\sigma{=}0.115$), with concentrated hot-spots along the scan direction. (b) Rectangular-spiral: "X"-shaped corner hot-spots ($\mu{=}0.890$, $\sigma{=}0.124$), indicating the highest activation variance among all strategies. (c) Proposed Fermat Spiral: markedly more uniform coverage ($\mu{=}0.621$, $\sigma{=}0.088$): the standard deviation is reduced by 24% vs. raster scan and 29% vs. rectangular-spiral, and mean activation concentration is 30% lower than raster scan, quantitatively confirming near-isotropic, direction-agnostic spatial modelling. Colour scale: 0 (dark blue) $\to$ 1 (yellow).
  • Figure 3: Qualitative comparison. Rows 1--3: Intra-modal synthesis on merged brain dataset---T1n$\to$T1c (metastasis), T2f$\to$T1c (meningioma), T2w$\to$T1c (glioma); orange boxes highlight tumour core ROIs. Row 4: Cross-modal CT$\to$MRI (SynthRAD2023); orange arrows indicate zoom-in panels at the ventricular wall. Numbers: SSIM.