Experimental insights into data augmentation techniques for deep learning-based multimode fiber imaging: limitations and success

TL;DR

The paper addresses data augmentation for deep learning-based multimode fiber imaging, where traditional digital augmentations and CGAN-based synthesis fail to respect the fiber's modal physics. It introduces a physical data augmentation method that digitally transforms organ labels while experimentally recording the corresponding speckles, yielding up to a 17% SSIM improvement under limited data. The findings highlight the necessity of incorporating light propagation physics into data preparation and point toward physics-informed generative models to reduce data collection while maintaining reconstruction fidelity, with implications for MMF endoscopy and other scattering-media imaging. The approach underscores that effective augmentation in optically complex systems must couple digital transformations with physically acquired outputs.

Abstract

Multimode fiber~(MMF) imaging using deep learning has high potential to produce compact, minimally invasive endoscopic systems. Nevertheless, it relies on large, diverse real-world medical data, whose availability is limited by privacy concerns and practical challenges. Although data augmentation has been extensively studied in various other deep learning tasks, it has not been systematically explored for MMF imaging. This work provides the first in-depth experimental and computational study on the efficacy and limitations of augmentation techniques in this field. We demonstrate that standard image transformations and conditional generative adversarial-based synthetic speckle generation fail to improve, or even deteriorate, reconstruction quality, as they neglect the complex modal interference and dispersion that results in speckle formation. To address this, we introduce a physical data augmentation method in which only organ images are digitally transformed, while their corresponding speckles are experimentally acquired via fiber. This approach preserves the physics of light-fiber interaction and enhances the reconstruction structural similarity index measure~(SSIM) by up to 17\%, forming a viable system for reliable MMF imaging under limited data conditions.

Figures (9)

• Figure 1: Experimental multimode fiber imaging setup showing the complete optical path from the laser source to the camera, including mirrors, lenses, polarizer, beam splitter, spatial light modulator, and the multimode fiber used for speckle acquisition.
• Figure 2: The architecture of conditional generative adversarial network
• Figure 3: Illustration of standard augmentation techniques applied to multimode fiber imaging. Simple transformations such as rotation, flipping, cropping, and brightness changes alter the input organ images and their corresponding speckle patterns.
• Figure 4: Reconstruction results for standard augmentation experiments. The model is trained on 5,000 original and augmented datasets. The reconstructed images and their average SSIM values indicate that conventional digital augmentations degrade image reconstruction quality in MMF imaging.
• Figure 5: Stepwise workflow of the CGAN-based data augmentation process for multimode fiber imaging. The figure outlines how the model is first trained on real speckles, then used to generate synthetic speckles from organ images, and finally retrained on a mixed dataset of real and synthetic samples to evaluate the impact on reconstruction quality.