Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

HistoSpeckle-Net: Mutual Information-Guided Deep Learning for high-fidelity reconstruction of complex OrganAMNIST images via perturbed Multimode Fibers

Jawaria Maqbool, M. Imran Cheema

TL;DR

The paper tackles the challenge of reconstructing structurally complex OrganAMNIST medical images from multimode-fiber speckle patterns, where conventional learning models struggle under data scarcity and fiber perturbations. It introduces HistoSpeckle-Net, a distribution-aware framework that combines differentiable histograms for a mutual-information loss with a multiscale SSIM objective and a Three-Scale Feature Refinement Module to recover high-fidelity images from speckle data. Key contributions include a Histogram Computation Unit for differentiable marginal and joint histograms, an MI-based loss that maximizes information transfer $I(y,G_3(x))$ by minimizing $H(y|G_3(x))$, and a computationally efficient TFRM that enables multiscale refinement and improved detail. The results show superior SSIM on OrganAMNIST compared to U-Net and Pix2Pix, with robustness to fiber bending and limited training data, advancing MMF imaging toward practical clinical deployment and suggesting applicability to color imaging and other scattering-media tasks.

Abstract

Existing deep learning methods in multimode fiber (MMF) imaging often focus on simpler datasets, limiting their applicability to complex, real-world imaging tasks. These models are typically data-intensive, a challenge that becomes more pronounced when dealing with diverse and complex images. In this work, we propose HistoSpeckle-Net, a deep learning architecture designed to reconstruct structurally rich medical images from MMF speckles. To build a clinically relevant dataset, we develop an optical setup that couples laser light through a spatial light modulator (SLM) into an MMF, capturing output speckle patterns corresponding to input OrganAMNIST images. Unlike previous MMF imaging approaches, which have not considered the underlying statistics of speckles and reconstructed images, we introduce a distribution-aware learning strategy. We employ a histogram-based mutual information loss to enhance model robustness and reduce reliance on large datasets. Our model includes a histogram computation unit that estimates smooth marginal and joint histograms for calculating mutual information loss. It also incorporates a unique Three-Scale Feature Refinement Module, which leads to multiscale Structural Similarity Index Measure (SSIM) loss computation. Together, these two loss functions enhance both the structural fidelity and statistical alignment of the reconstructed images. Our experiments on the complex OrganAMNIST dataset demonstrate that HistoSpeckle-Net achieves higher fidelity than baseline models such as U-Net and Pix2Pix. It gives superior performance even with limited training samples and across varying fiber bending conditions. By effectively reconstructing complex anatomical features with reduced data and under fiber perturbations, HistoSpeckle-Net brings MMF imaging closer to practical deployment in real-world clinical environments.

HistoSpeckle-Net: Mutual Information-Guided Deep Learning for high-fidelity reconstruction of complex OrganAMNIST images via perturbed Multimode Fibers

TL;DR

The paper tackles the challenge of reconstructing structurally complex OrganAMNIST medical images from multimode-fiber speckle patterns, where conventional learning models struggle under data scarcity and fiber perturbations. It introduces HistoSpeckle-Net, a distribution-aware framework that combines differentiable histograms for a mutual-information loss with a multiscale SSIM objective and a Three-Scale Feature Refinement Module to recover high-fidelity images from speckle data. Key contributions include a Histogram Computation Unit for differentiable marginal and joint histograms, an MI-based loss that maximizes information transfer by minimizing , and a computationally efficient TFRM that enables multiscale refinement and improved detail. The results show superior SSIM on OrganAMNIST compared to U-Net and Pix2Pix, with robustness to fiber bending and limited training data, advancing MMF imaging toward practical clinical deployment and suggesting applicability to color imaging and other scattering-media tasks.

Abstract

Existing deep learning methods in multimode fiber (MMF) imaging often focus on simpler datasets, limiting their applicability to complex, real-world imaging tasks. These models are typically data-intensive, a challenge that becomes more pronounced when dealing with diverse and complex images. In this work, we propose HistoSpeckle-Net, a deep learning architecture designed to reconstruct structurally rich medical images from MMF speckles. To build a clinically relevant dataset, we develop an optical setup that couples laser light through a spatial light modulator (SLM) into an MMF, capturing output speckle patterns corresponding to input OrganAMNIST images. Unlike previous MMF imaging approaches, which have not considered the underlying statistics of speckles and reconstructed images, we introduce a distribution-aware learning strategy. We employ a histogram-based mutual information loss to enhance model robustness and reduce reliance on large datasets. Our model includes a histogram computation unit that estimates smooth marginal and joint histograms for calculating mutual information loss. It also incorporates a unique Three-Scale Feature Refinement Module, which leads to multiscale Structural Similarity Index Measure (SSIM) loss computation. Together, these two loss functions enhance both the structural fidelity and statistical alignment of the reconstructed images. Our experiments on the complex OrganAMNIST dataset demonstrate that HistoSpeckle-Net achieves higher fidelity than baseline models such as U-Net and Pix2Pix. It gives superior performance even with limited training samples and across varying fiber bending conditions. By effectively reconstructing complex anatomical features with reduced data and under fiber perturbations, HistoSpeckle-Net brings MMF imaging closer to practical deployment in real-world clinical environments.

Paper Structure

This paper contains 8 sections, 16 equations, 6 figures.

Table of Contents

  1. Introduction
  2. Experimental Setup
  3. Histogram computation unit
  4. Deep Learning Framework
  5. Three-Scale Feature Refinement Module (TFRM)
  6. Loss function
  7. Reconstruction results
  8. Conclusion

Figures (6)

  • Figure 1: Experimental setup for data collection corresponding to three different multimode fiber (MMF) configurations. LD: Laser diode, C: Collimator, L: Lens, M: Mirror, BS: Beam splitter, P: Linear polarizer, SLM: Spatial light modulator, CCD: Camera, MMF: Multimode fiber, Cf: Fiber configuration.
  • Figure 2: Comparison of normalized intensity histograms for (a) speckle input,(b) true label,(c) initially generated image, and (d) final generated image. The original histograms (shaded area) and smooth differentiable histograms (blue line) are shown. Histogram alignment with ground truth (b) improves from the (c) initial to the (d) final generated images, indicating learning progression.
  • Figure 3: An overview of computing smooth marginal histograms and the joint histogram for both the generated and ground truth images using a histogram computation unit. Each image is passed through a bank of 256 Gaussian kernels to generate kernel response maps. The kernel responses are then summed and normalized to produce smooth histograms (green curves), which closely match the original discrete histograms (light blue bars). These maps are also flattened into matrices $\tilde{W}_A$ and $\tilde{W}_B$, representing smooth assignments to histogram bins. The flattened matrices are multiplied and normalized to produce the final joint histogram.
  • Figure 4: The complete architecture of HistoSpeckle-Net
  • Figure 5: Reconstruction results for a fixed fiber and for different numbers of training samples.
  • ...and 1 more figures