Differentiable Microscopy Designs an All Optical Phase Retrieval Microscope

Abstract

Designing new optical systems from the ground up for microscopy imaging tasks such as phase retrieval, requires substantial scientific expertise and creativity. To augment the traditional design process, we propose differentiable microscopy ($\partialμ$), which introduces a top-down design approach. Using all optical phase retrieval as an illustrative example, we demonstrate the effectiveness of data-driven microscopy design through $\partialμ$. Furthermore, we conduct comprehensive comparisons with existing all-optical phase retrieval methods, showcasing the consistent superiority of our learned designs across multiple datasets, including biological samples. To substantiate our ideas, we experimentally validate the functionality of one of the learned designs, providing a proof of concept. The proposed differentiable microscopy framework supplements the creative process of designing new phase microscopy systems and may be extended to other similar applications in optical design.

Figures (17)

• Figure 1: Differentiable microscopy ($\partial \mu$) based all-optical phase to intensity conversion.(A) Problem formulation as a learning task and model implementation pipeline in differentiable microscopy. (B) The complex-valued linear CNN architecture that was implemented mimicking optical constraints for linear phase retrieval. Note that there are no non-linear activations between convolutions. (C) Learnable Fourier filter (LFF) based design. Here, $f$ denotes the focal length of the lens used to implement the 4-f system. (D) Diffractive deep neural network based design (D2NN). Note that D2NN is very compact and less than $50$$\mu m$ (i.e. $51.7\lambda$, where $\lambda$ is the operating wavelength) thick.
• Figure 2: 4-f system, where the distance between the input plane ($P_1$) and the image plane ($P_3$) equals to $4 \times f$. Here, $L_1, L_2, L_3$ are identical lenses each having a focal length $f$, $P_1, P_2, P_3$ are the input plane, Fourier plane and the image plane, respectively, and $S$ is a point source.
• Figure 3: Diffraction by an aperture in a planar screen.
• Figure 4: Diffraction of light waves by a neuron of a D2NN layer.
• Figure 5: Overall qualitative results.(A) The amplitude (top row) and phase (bottom row) of the input field to the system for each dataset. B The results of the complex-valued linear CNN for the phase-to-intensity conversion task on each test dataset. The output intensity maps are visually similar to the input phase maps, suggesting a linear model can learn to convert phase to intensity for a given dataset. C1, C2, C3 Show the reconstruction results by the best performing LFF, $\phi$-LFF, and $\phi$-LRF learned for each dataset, respectively. Similarly, D1, D2, E1, E2 show the respective results for the GPC, $\phi$-GPC, D2NN, and $\phi$-D2NN. F1 Shows the amplitude (top row) and phase (bottom row) of the learned LFFs for each dataset. F2, F3 Shows the phase of the learned $\phi$-LFF and $\phi$-LRF for each dataset respectively. G1 Shows the amplitude (top row) and phase (bottom row) of the learned GPC filters for each dataset. G2 Shows the phase of the learned $\phi$-GPC filter for each dataset. H1 Shows a comparison of $\partial\mu$ architectures on different datasets. Our approach consistently outperforms the GPC Gluckstad2009 for all datasets except the HeLa$[0,\pi]$ dataset, where it performs on-par with the GPC. The complex-valued CNN sets the empirical upper bounds for each dataset. H2 Shows the performance of each $\partial\mu$ and GPC architecture with the trainable number of parameters. The $\phi$-LRF achieves comparable/ superior performance for the MNIST $[0,2\pi]$ and HeLa $[0, \pi]$ datasets with a significantly lower number of trainable parameters compared to the LFFs and $\phi$-D2NN architectures. Even though the GPC shows comparable performance with the $\phi$-LRF for the HeLa $[0, \pi]$ dataset, the $\phi$-LRF shows a significant improvement in performance for the MNIST $[0, 2\pi]$ dataset.