DeepPD: Joint Phase and Object Estimation from Phase Diversity with Neural Calibration of a Deformable Mirror

TL;DR

DeepPD addresses the challenge of recovering both sample structure and high-order phase aberrations in fluorescence microscopy from a minimal five-image phase-diversity sequence. It combines neural representations for the object and phase with a learned deformable-mirror calibration model, all within a differentiable imaging framework, enabling joint estimation of $O$ and $\phi$ from the set $\{I_k\}_{k=0}^4$ where $I_k = O * | \mathcal{F}^{-1}(P e^{i(\phi+\psi_k)})|^2$. A neural mirror model predicts and inverts the DM response, addressing nonlinearities and actuator couplings that limit linear calibrations. On calibration slides and immunolabeled PtK2 cells, DeepPD outperforms Gauss–Newton, Poisson, and purely neural baselines, delivering more accurate object reconstructions, higher-frequency content, and robustness to aberrations up to RMS wavefront distortions of about $350\,\mathrm{nm}$. The approach reduces acquisition time, supports near real-time correction for moderate-sized images, and points toward rapid, high-fidelity phase retrieval and image correction in live-cell imaging contexts.

Abstract

Sample-induced aberrations and optical imperfections limit the resolution of fluorescence microscopy. Phase diversity is a powerful technique that leverages complementary phase information in sequentially acquired images with deliberately introduced aberrations--the phase diversities--to enable phase and object reconstruction and restore diffraction-limited resolution. These phase diversities are typically introduced into the optical path via a deformable mirror. Existing phase-diversity-based methods are limited to Zernike modes, require large numbers of diversity images, or depend on accurate mirror calibration--which are all suboptimal. We present DeepPD, a deep learning-based framework that combines neural representations of the object and wavefront with a learned model of the deformable mirror to jointly estimate both object and phase from only five images. DeepPD improves robustness and reconstruction quality over previous approaches, even under severe aberrations. We demonstrate its performance on calibration targets and biological samples, including immunolabeled myosin in fixed PtK2 cells.

Figures (6)

• Figure 1: DeepPD method overview. The schematic depicts the concept of estimating the unknown object and phase aberration based on phase diversity. The estimated sample and estimated phase aberrations are represented by neural representations. The phase diversities are obtained from the input voltages based on a pretrained mirror model. By convolving the current object estimate with PSFs based on the current phase aberration estimate and the input phase diversities we obtain an estimate of the aberrated image and the four phase-diversity images (i.e., a total of five images). Note that the aberrated image only includes the estimated phase aberration and no additional phase diversity (indicated by the flat phase). The obtained estimated acquisition is compared to the acquired images to calculate the loss, which is used to train the neural representations. Scale bars: $1\um$.
• Figure 1: Comparison of runtimes and corresponding achieved DCT norm of the object estimate for the Gaussian, Poisson, and DeepPD methods for a 512$\times$512 image region of the myosin dataset shown in Fig. \ref{['fig:myosin']} in the main text.
• Figure 2: Learned mirror calibration. (a) Applying voltage to each of the 52 actuators (left) shapes the mirror's deformable membrane. Numbers indicate the mirror actuator indices. Assuming linearity, the mirror's response to increasing the voltage for a single actuator can be described by an influence function (right). (b) Phase-diversity images of a training data set. Data is recorded on a bead sample. The phase aberration is estimated from a phase-diversity acquisition data set consisting of the aberrated image and four phase-diversity images (top). The input voltages (bottom left) together with the obtained phase estimates (bottom center) from the neural representations (Fig. \ref{['fig:method-overview']}) constitute one training data pair for the mirror model. The object estimate is not used for training the mirror model, but is shown here for illustration (bottom right). Scale bars: $2\um$. (c) Network architecture of the mirror model for predicting (non-linear) phase response from voltage input (top) and obtaining required input voltages for a desired phase response (bottom). Details of the architectures are described in the \ref{['sec:methods']}. (d) Wavefront induced by the deformable mirror for two sets of applied voltages. The left column shows the input voltages applied to the 52 mirror actuators. Columns 2--4 compare the expected wavefront by linear combination of the influence functions, predicted mirror response by the trained neural-network mirror model, and SH measurements of the phase aberration. Note that the SH measurements are shown in a zonal representation, and the pupil diameter cannot be matched directly.
• Figure 3: Performance comparison of various methods on the Argo-HM calibration slide. (a) Recorded microscope images. Left to right: Aberrated image, four phase-diversity images, and the unaberrated image for comparison. Scale bars: $2\um$. (b) Object estimate (top row), phase estimate (middle row) and corresponding PSF (bottom row) obtained by various methods (left to right: Gauss--Newton algorithm, Poisson algorithm, neural representations assuming a linear mirror response (NNRs), and neural representations using the trained mirror model (DeepPD)). The last column shows an object estimate obtained by Richardson--Lucy deconvolution (20 iterations) of the unaberrated image, flat phase and corresponding PSF for comparison. Scale bars: $2\um$ (object estimates), $1\um$ (PSFs).
• Figure 4: Performance comparison of various methods on a biological sample of myosin filaments (fixed PtK2 cell immunolabeled against myosin heavy chain). (a) Recorded images. Left to right: Aberrated image, zoom-in of the aberrated image (region of interest indicated by the white box in the full image), four phase-diversity images, and the unaberrated image for comparison. Scale bars: $20\um$ (first image), $2\um$ (others). (b) Object estimate (top row), phase estimate (middle row) and corresponding PSF (bottom row) obtained by various methods (left to right: Gauss--Newton algorithm, Poisson algorithm, neural representations assuming a linear mirror response (NNRs), and neural representations using the trained mirror model (DeepPD)). The last column shows an object estimate obtained by Richardson--Lucy deconvolution (20 iterations) of the unaberrated image, flat phase and corresponding PSF for comparison. Scale bars: $2\um$.