FLASHμ: Fast Localizing And Sizing of Holographic Microparticles

TL;DR

A two-stage neural network architecture is designed, FLASH, to detect small particles from holograms with large sample depths up to 20cm, comparable to the standard reconstruction-based approaches while operating on smaller crops, and providing roughly a 600-fold speedup.

Abstract

Reconstructing the 3D location and size of microparticles from diffraction images - holograms - is a computationally expensive inverse problem that has traditionally been solved using physics-based reconstruction methods. More recently, researchers have used machine learning methods to speed up the process. However, for small particles in large sample volumes the performance of these methods falls short of standard physics-based reconstruction methods. Here we designed a two-stage neural network architecture, FLASH$μ$, to detect small particles (6-100$μ$m) from holograms with large sample depths up to 20cm. Trained only on synthetic data with added physical noise, our method reliably detects particles of at least 9$μ$m diameter in real holograms, comparable to the standard reconstruction-based approaches while operating on smaller crops, at quarter of the original resolution and providing roughly a 600-fold speedup. In addition to introducing a novel approach to a non-local object detection or signal demixing problem, our work could enable low-cost, real-time holographic imaging setups.

Figures (14)

• Figure 1: Motivation and overview. A. Pictorial representation of the inverse problem. B. Schematic diagram of our approach: FLASHµ. C. Current ML-based approaches to hologram reconstruction rely on object detection, which limits their applicability to small sample depths (20-40 mm) where fringe patterns are localized. D. The contrast of a particle is approximately proportional to its cross-sectional area, making small particles extremely hard to detect because of their low contrast. E. With smaller particles $\leq$ 20 µm and increased sample depth, fringe patterns extend over almost the entire image and many particles' signals need to be demixed. F. Comparison between a synthetic hologram and a real hologram with ground truth (GT) overlaid. Real holograms have much lower signal-to-noise ratio.
• Figure 2: A schematic diagram describing the standard reconstruction-based method for particle extraction.
• Figure 3: Stage I: CSFM. The hologram $\mathcal{H}$ is ifted to a multi-channel image, goes through several Dilated Fourier (DF) and Dilated Convolutional (DC) blocks, and rojected to Weighted Hologram $\mathcal{H}_w$. In the DF block, the input signal $u(\mathbf{x})$ is Fourier transformed $\mathcal{F}$, performs pointwise multiplication with the Fourier weights (w$_1$,w$_2$,…) convolved with kernel , and taken back to pixel space with $\mathcal{F}^{-1}.$ The bottom skip-connection contains a convolutional kernel . The DC inputs $v(\mathbf{x})$ which goes through three dilated convolutions with rates 1,3 and 9, and combines with a identity skip-connection. Stage II: Detection. The concatenated hologram, weighted-hologram $(\mathcal{H}, \mathcal{H}_w)$ pair is fed into the U-net. Localized fringes in $\mathcal{H}_w$ help the U-net to pinpoint (as $(x,y)$ Gaussian blobs) the particle with limited receptive field. Additionally, depth $z$ and size $d$ (diameter) are regressed as separate channels.
• Figure 4: Toy example illustrating how NNs can overcome the limits of reconstruction-based methods in terms of sensor size and resolution. A: Two particles (6 µm and 100 µm), placed 200 mm from the camera. B: Resulting $512\times 512$ hologram. C: Standard reconstruction at $z = 200$ mm. D: Only $100$ µm particle is detected from the reconstruction. E: U-net (shown here) and FLASHµ trained on toy dataset (I) reliably detect both particles.
• Figure 5: Quantitative evaluation of FLASHµ and baselines (U-net and SVMs) on 384$\times 384$ (S)ynthetic holograms. A: Precision over recall for the synthetic test dataset. B: F1 score as function of distance from camera $z$. C: F1 score as function of particle diameter $d$. D: F1 score as function of particle number density. E, F: Comparison of median and interquartile of absolute error in $z$ with respect to $z$ (E) and $d$ (diameter) (F).