NeOTF: Guidestar-free neural representation for broadband dynamic imaging through scattering

TL;DR

Dynamic imaging through scattering media faces rapid speckle decorrelation and low signal-to-noise ratios. NeOTF introduces a guidestar-free approach that learns a spatially continuous OTF phase $\phi_S(f_x,f_y)$ via an implicit neural representation, optimized against multi-frame speckle data through a self-supervised loss $\mathcal{L}$ in the Fourier domain. The method demonstrates robust dynamic imaging under broadband illumination up to $\sim$300 nm and extremely low SNR, leveraging the spatio-temporal memory effect to track time-varying OTFs without guidestars, and shows favorable computational efficiency via coordinate-based sampling and partial frequency-domain coverage. These results suggest NeOTF as a practical, scalable solution for high-fidelity, dynamic imaging through scattering across static and dynamic media, with open-source code and models.

Abstract

Dynamic imaging through time-varying scattering media is ubiquitous in real-world settings, yet it remains a defining unsolved problem as rapid spatiotemporal fluctuations overwhelm standard reconstruction pipelines that often rely on speckles with high signal-to-noise ratio. Existing approaches fall into two categories. Guidestar-based methods employ a guidestar to recover the system transfer function; however, in dynamic media, the speckle decorrelates rapidly, making the calibration quickly invalid. Guidestar-free methods infer information from speckle statistics, but rapid changes and noise often break phase retrieval. To overcome these limitations, we introduce NeOTF, a guidestar-free and neural-representation-based OTF retrieval method that enables dynamic imaging through time-varying scattering media. By optimizing this neural representation with only a few speckle images from unknown objects, NeOTF robustly retrieves the system's OTF without a guidestar. We experimentally demonstrate robust dynamic imaging through scattering with NeOTF at extremely low signal-to-noise ratio and broadband incoherent illumination (up to 300 nm spectral bandwidth) scenarios, and we numerically validate its dynamic imaging performance in time-varying scattering media leveraging spatio-temporal memory effect. Finally, we discuss and analyze its computational efficiency and generalization capabilities across anisotropic scattering media. These results establish NeOTF's promise as a practical and robust solution for dynamic imaging through scattering media. Open-sourced code and models are available at https://github.com/Xia-Research-Lab/NeOTF.

Figures (9)

• Figure 1: Illustration of main methods for imaging through scattering media. (a) Traditional single-shot speckle correlation method for imaging through a scattering medium. (b) Guidestar-based imaging method. The objects are directly recovered by deconvolution with the PSF calibrated from the point source which acts as a guidestar. (c) Our proposed image-guided NeOTF-based imaging method. The phase of the OTF is retrieved from the unknown speckles using iterative optimization. The image can be directly restored from the speckle by inverse filtering.
• Figure 2: Overiew of our proposed NeOTF-enabled imaging framework. (a) Pipeline of NeOTF optimization and image reconstruction. The optimization begins with multi-frame images as input, and the reconstructed object intensity image(s) as the output. The yellow solid line indicates the forward pass in the image reconstruction workflow, while the blue dotted line depicts backward pass mainly composed of the OTF retrieval. (b) The network architecture of NeOTF. It consists of a frequency encoding layer, two hidden layers with SIREN activation functions, and an output layer with arctan activation function. The input is coordinates of $(f_x,f_y)$, and the output $\phi_{S_{i}}(f_x,f_y)$ is the calculated phase corresponding to coordinates $(f_x,f_y)$ at the $i$-th epoch.
• Figure 3: Experimental setup and illumination source characterization (a) Experimental setup of non-invasive imaging through a scattering medium. In our experimental demonstration, we use three different incoherent illumination sources, the first one is a rotating ground glass (not shown in the figure) combined with a He-Ne laser, which is used to generate a pseudothermal light source; the second one is a red monochromatic LED; and the third one is a white LED. A Digital Micromirror Device (DMD) is used to display the dynamic object, which reflects the light through a diffuser and create speckle image captured by a camera. (b) Spectral intensity distribution of the sources in (a) measured by an optical fiber spectrometer.
• Figure 4: Experimental validation of NeOTF's robustness in low-SNR and incoherent illumination conditions. Reconstructions are compared under several different incoherent illumination sources and exposure times: a He-Ne laser with rotating diffuser as a pseudothermal light source (1 nm / 500 ms), a red LED (15 nm / 500 ms), and a white-light LED (300 nm / 500 ms and 50 ms), yielding the autocorrelation's SNRs of 28.74 dB, 25.91 dB, 17.42 dB, and 14.89 dB, respectively. Each panel shows the raw speckle image measured through scattering, its 2D autocorrelation image, the reconstruction results obtained based on conventional methods including the traditional HIO-ER and MORE, and the result from our proposed NeOTF method, with the ground truth (GT) image on the right. autocor.: 2D autocorrelation of the speckle image.
• Figure 5: Dynamic imaging of a C. elegans embryogenesis and chiral rotation through static scattering medium. (a) Illustration for dynamic imaging through a static diffuser. The OTF of the time-invariant imaging system is retrieved from initial 5 frames of speckles in a short time period ($\Delta T_0$). Subsequent dynamic scenes are all reconstructed using the restored NeOTF ($\theta_0$). (b) Reconstructed results from the dynamic speckles from simulated guidestar-based reconstruction and the proposed NeOTF method. "GT" indicates ground truth images. "Speckle" indicates low SNR speckle images collected through scattering. "guidestar" indicates the results recovered from a simulated OTF measurement using a synthetic guidestar. "NeOTF" shows the images reconstructed by the proposed guidestar-free based NeOTF framework.