Physics-guided foundation model for universal speckle removal in ultrathin multimode fiber imaging

TL;DR

This work tackles the challenge of speckle-limited imaging through ultrathin multimode fibers by introducing SCNet, a physics-guided foundation model that combines a Mixture of Experts with material-aware routing, Haar wavelet frequency decomposition, and a dual-domain curriculum to universally suppress speckle without target-specific retraining. The framework employs a dual-fiber holographic endoscope for stand-off imaging and validates performance across diverse materials and biological tissues, achieving depth-invariant reconstructions and high spatial resolution (e.g., 5.66 lp/mm on paper) while enabling real-time inference through multi-teacher distillation that yields 60 FPS. Key innovations include the integration of a Haar DWT-based frequency domain operator, a spectral gating mechanism within a MoE, and a two-stage loss that first enforces frequency-domain consistency (CDPO-Loss) and then spatial fidelity with a structural-consistency term. Collectively, the results demonstrate universal generalization across scattering regimes, enhanced organ imaging in rabbits under ultrathin collection constraints, and a practical path toward speckle-free ultrathin endoscopy in confined spaces, with potential extensions to other computational-imaging modalities.

Abstract

Ultrathin multimode fibers (MMFs) promise endoscopes with hair-scale diameters for accessing sub-millimeter anatomy, but in MMF far-field imaging the required small collection aperture drives speckle-dominated measurements that rapidly degrade image fidelity. Here we present Speckle Clean Network (SCNet), a physics-guided foundation model for universal speckle removal that makes photon-limited, single-fiber collection compatible with high-fidelity reconstruction across diverse scattering conditions without target-specific retraining. SCNet combines a Mixture of Experts (MoE) architecture with material-aware routing, wavelet-based frequency decomposition to separate structure from speckle across sub-bands, and a curriculum-style optimization that progressively enforces spectral consistency before spatial fidelity. Using an ultrathin dual-fiber holographic probe, we deliver wavefront-shaped illumination through one s and collect backscattered photons through a parallel MMF. We validate SCNet on 3D plastic objects over varying working distances, resolve 5.66 lp/mm on a paper USAF target, and restore fine structures on leaves and metal surfaces. On rabbit heart and kidney tissues, SCNet improves recovery of low-contrast anatomical texture under the same ultrathin collection constraint. We further compress SCNet through multi-teacher distillation to reduce computation while preserving reconstruction quality, enabling inference at 60 FPS. This work effectively decouples image quality from probe size, establishing a speckle-free ultrathin endoscopy for stand-off imaging in confined spaces.

Figures (9)

• Figure : Fig.1|SCNet speckle suppression model architecture for ultrathin multimode-fiber endoscopic imaging.a, Schematic of the holographic endoscopic system. A DMD shapes the wavefront to focus light through an illumination MMF onto a distant object. Backscattered signal is collected via a second parallel MMF to a bucket detector. b, Architecture of the SCNet model. The network features a MoE block controlled by a gating mechanism that dynamically weights material-specific experts to handle diverse scattering properties. c, Dual-domain curriculum learning strategy. The model is trained first by optimizing for frequency domain consistency using wavelet transforms to learn noise-invariant features, followed by spatial domain fine tuning to recover high-frequency morphological details. d, Representative SCNet reconstructions for a leaf and mammalian tissues (rabbit heart and kidney), showing the raw speckled images (top) and the corresponding SCNet outputs (bottom).
• Figure : Fig.2|Depth-invariant reconstruction of 3D polymeric phantoms via dynamic expert routing.a, Comparison of raw and processed images of Lego minifigures at different working distances (3, 4, 5 cm). Raw images (LQ) were captured using a single-core ultrafine multimode fiber with a core diameter of 100 µm; processed images (Ours) represent results obtained via the SCNet model; ground truth images (GT) were acquired using a fiber bundle comprising three fibers with a core diameter of 1000µ m. The enlarged region in the lower right corner of the original image demonstrates the detail recovery in the eye and eyeglass areas. b-d, Quantitative comparison of different models across three metrics-Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), and Root Mean Square Error (RMSE) at each working distance shown in (a). Higher SSIM and PSNR values indicate better performance, while lower RMSE values denote superior performance. e-g, Intensity distribution curves extracted along the facial region (green dashed line) at working distances of 3 cm (e), 4 cm (f), and 5 cm (g). Black, gray, and red curves represent intensity distributions for GT, LQ, and SCNet speckle-removed output, respectively. h, Based on the intensity distributions in (e-g), the coefficient of determination (R²) quantifies the goodness of fit between SCNet outputs and GT. i-k, Overall performance comparison of different models on the full plastic sample test set, presented as box plots for SSIM (i), PSNR (j), and RMSE (k). Box plots show the median (box line), interquartile range (box width), and 1.5 times the interquartile range (whiskers).
• Figure : Fig.3|High-resolution imaging on absorbent paper via spectral consistency learning.a, Schematic of the standard USAF-1951 resolution target. b-e, Reconstruction of Group 2-3 Elements at working distances of 9, 8, 7, and 6 mm. Top row: SCNet outputs showing resolved line pairs. Bottom row: Corresponding vertical line sections (yellow dashed lines), used to determine the maximum resolvable spatial frequency based on peak-to-valley contrast. f, Maximum resolvable spatial frequency (line pairs/mm) achieved by SCNet compared to baseline models (CGNet, NAFNet, KBNet, Restormer, SwinIR, and BM3D) across all distances. SCNet consistently resolves 5.66 lp/mm (Group 2, Element 4). g, Macroscopic reconstruction of Group 0 Elements 4-6 from the USAF target; green dashed box marks the zoomed in region. h,i, Line-pair intensity profiles along the Element 6 horizontal and vertical lines (blue and red dashed lines in g) for all methods, including GT and LQ. j, Comparison of the goodness-of-fit ($R^2$) between each method and the GT profiles in h and i. k, Quantitative evaluation of speckle reduction performance across models on the images shown in g, including SSIM, PSNR, and RMSE metrics. l-n, Overall performance comparison on the entire paper test set, presented as box plots showing SSIM (l), PSNR (m), and RMSE (n).
• Figure : Fig.4|Complex structure restoration across biological and metal surfaces.a, Schematic of the leaf sample, with the red box indicating the imaging acquisition area. b, Reconstruction of leaf vasculature. c,d, Quantitative performance (PSNR, RMSE, SSIM, $R^2$) on leaf samples. e, Line intensity profiles across a secondary vein. f-h, Statistical performance distributions (SSIM, PSNR, RMSE) for the complete leaf test set. i, Schematic of the metallic sample showing engraved characters, with the green dashed box marking the magnified “A” region and the red dashed line showing the intensity extraction path along the letter “S”. j, Metal surface speckle removal. Top row shows full-field reconstructions; bottom row shows magnified views of the “A” region for each method. k,l, Quantitative performance (PSNR, RMSE, SSIM, $R^2$) on metal samples. m, Line intensity profile across the sharp edge of the letter “S”. n-p, Statistical performance distributions for the complete metal test set.
• Figure : Fig.5|Biological organ imaging and computational optimization.a,b, Representative reconstructions of rabbit heart (a) and kidney (b) tissues, shown as GT, SCNet output and LQ; the second row shows magnified views of the green dashed regions. c-e, Statistical performance comparison of SCNet with competing methods for biological test sets, showing as SSIM (c), PSNR (d), and RMSE (e). f, Comparison of image processing performance metrics for MSE loss function, CDPO loss function, and CLAHE histogram equalization, shown as RMSE, PSNR and SSIM. g, Comparison of computational complexity (GMACs) and inference speed (FPS) across all models. h, Average performance summary across five material test sets; vertical axis shows average SSIM, horizontal axis shows average RMSE and bubble size indicates average PSNR.