FPM-WSI: Fourier ptychographic whole slide imaging via feature-domain backdiffraction

TL;DR

This work reports a computational framework based on feature-domain backdiffraction to realize full-FOV, stitching-free FPM reconstruction, and presents effective elimination of vignetting artifacts, and finds its great potential to recover the data with a lower overlapping rate of spectrum.

Abstract

Fourier ptychographic microscopy (FPM), characterized by high-throughput computational imaging, theoretically provides a cunning solution to the trade-off between spatial resolution and field of view (FOV), which has a promising prospect in the application of digital pathology. However, block reconstruction and then stitching has currently become an unavoidable procedure due to vignetting effects. The stitched image tends to present color inconsistency in different image segments, or even stitching artifacts. In response, we reported a computational framework based on feature-domain backdiffraction to realize full-FOV, stitching-free FPM reconstruction. Different from conventional algorithms that establish the loss function in the image domain, our method formulates it in the feature domain, where effective information of images is extracted by a feature extractor to bypass the vignetting effect. The feature-domain error between predicted images based on estimation of model parameters and practically captured images is then digitally diffracted back through the optical system for complex amplitude reconstruction and aberration compensation. Through massive simulations and experiments, the method presents effective elimination of vignetting artifacts, and reduces the requirement of precise knowledge of illumination positions. We also found its great potential to recover the data with a lower overlapping rate of spectrum and to realize automatic blind-digital refocusing without a prior defocus distance.

Figures (6)

• Figure 1: Color image of a pathology slide via FPM-WSI. (a) sample slide. (b1), (b2) and (b3) show first 25 images of FPM datacube for red, green and blue colors illumination. Vignetting effect is prominent. (c) $3.3 \times 3.3 \text{ mm}^2$ Full-FOV image of a human colorectal carcinoma section created by fusing reconstructions of R/G/B channels (refer to https://www.gigapan.com/gigapans/233966); (d1) and (e1) are reconstructions of two ROIs marked in (c), with a size of $650 \times 650$ pixels; (d2,e2) and (d3,e3) are corresponding images captured by a color image sensor using a $\times 20$ and $\times 4$ objective lens for comparison.
• Figure 2: FPM-SIM platform setup. (a) Overall architecture of FPM-SIM platform generally consisting of microscopic imaging system, automatic control system and a host computer; (b1) $19 \times 19$ programmable LED array for sample illumination; (b2) Packaged appearance of LED array with the central LED lightning; (b3) $z$ axis driver holding the objective lens for autofocusing; (b4) $x-y$ axis displacement stage for mechanical movement of a batch of 4 slides; (c) Optical path diagram of the microscope. (d) Flowchart for FPM-WSI reconstruction using feature-domain backdiffraction, involving 6 steps. Step 1: the model generates a series of predicted images based on current estimation of parameters including the comlex amplitude of the sample and pupil function. Step 2: the predicted images and their corresponding observed images are filtered by the feature extractor, producing the feature maps. Step 3: the feature-domain error between model prediction and the observations is calculated. Step 4: the error is back-diffracted to yield the complex-gradient. Step 5: the complex-gradient is managed by the optimizer with potential first-order and second-order moments. Step 6: the model parameters are updated.
• Figure 3: Reconstructions for lower overlapping rate of spectrum. (a1-a2) and (b1-b2) are reconstructed amplitudes and their spectrum with simulated overlapping rate of 22%. (c1-c2) lists the average value of PSNR and SSIM for two simulations of overlapping rates. (d1) and (d2) plot PSNR and SSIM score for 500 groups of simulation study, 2% Gaussian noise was added. The LED height controls the overlapping rates. (e) and (f) show experimental results using AS-EPRY and FPM-WSI respectively when the overlapping rate of spectrum is 22.47%.
• Figure 4: Comparison of experimental robustness for conventional FPM algorithm and FPM-WSI. (a) Simulated LED position shift in the LED array. (b-d) are simulated ground truth, reconstructed amplitude and corresponding Fourier spectrum. (e) lists the value of PSNR and SSIM for two methods. (f1,f2) plot the value of PSNR and SSIM for 500 groups of simulation with different degrees of LED position shift. (g) Raw data obtained with the illumination of 12 × 12 LEDs, and obvious vignetting effect can be found in the central 4 × 4 images. (h) Magnified view of ROI in the raw data. (i) Fourier spectrum of reconstruction using AS-EPRY and FPM-WSI. (j1,j2) show the reconstructed amplitude of USAF target and magnified view of ROI. (k) plots the quantitative profile along line $l_1$ and $l_2$ respectively. Scale bars in (h) and (j2) denote 14 $\upmu$m.
• Figure 5: Embedded pupil function recovery and digital refocusing for FPM reconstruction. (a1) Stitched reconstruction for a USAF target consisting of 16 image segments. (a2) Zoomed-in image of the region marked by the yellow box. (a3) Reconstructed spatially varying pupil functions for each segment. (b1) Central brightfield raw image of a defocused USAF target. (b2) Reconstructed results using AS-EPRY and FPM-WSI. (b3) and (c) are reconstructed pupil function corresponding to (b1) and its Zernike smoothed output. (d) plots the first 13 coefficients of the Zernike polynomial listed by fringe index.