PNR: Physics-informed Neural Representation for high-resolution LFM reconstruction

TL;DR

PNR (Physics-informed Neural Representation), a method for high-resolution LFM reconstruction that significantly enhances performance and integrates a physics-informed aberration correction strategy that optimizes Zernike polynomial parameters during optimization, thereby reducing the information loss caused by aberrations and improving spatial resolution.

Abstract

Light field microscopy (LFM) has been widely utilized in various fields for its capability to efficiently capture high-resolution 3D scenes. Despite the rapid advancements in neural representations, there are few methods specifically tailored for microscopic scenes. Existing approaches often do not adequately address issues such as the loss of high-frequency information due to defocus and sample aberration, resulting in suboptimal performance. In addition, existing methods, including RLD, INR, and supervised U-Net, face challenges such as sensitivity to initial estimates, reliance on extensive labeled data, and low computational efficiency, all of which significantly diminish the practicality in complex biological scenarios. This paper introduces PNR (Physics-informed Neural Representation), a method for high-resolution LFM reconstruction that significantly enhances performance. Our method incorporates an unsupervised and explicit feature representation approach, resulting in a 6.1 dB improvement in PSNR than RLD. Additionally, our method employs a frequency-based training loss, enabling better recovery of high-frequency details, which leads to a reduction in LPIPS by at least half compared to SOTA methods (1.762 V.S. 3.646 of DINER). Moreover, PNR integrates a physics-informed aberration correction strategy that optimizes Zernike polynomial parameters during optimization, thereby reducing the information loss caused by aberrations and improving spatial resolution. These advancements make PNR a promising solution for long-term high-resolution biological imaging applications. Our code and dataset will be made publicly available.

Figures (11)

• Figure 1: Quantitative and qualitative comparison of four loss functions used for image reconstruction (the lower half is the Fourier spectrum of the reconstruction results). The highlighted areas in the spatial and frequency domain of center view indicate that spectral cutoff and defocus in LFM can significantly lead to the loss of high-frequency information. Compared to other loss functions, FFT Loss effectively leverages multi-angular information to recover missing high-frequency details.
• Figure 2: Overview of the framework and training loss. Our framework, named PNR, contains four components: explicit feature planes, a two-layer MLP and digital adaptive optics (DAO) module. The yellow dashed box illustrates the actual light field imaging process using 2pSAM.
• Figure 3: Schematic diagram of sequential reconstruction. The sequential reconstruction process begins with a random initialization of Frame 1, followed by reconstruction (92.1s) and fine-tuning of subsequent frames (3.4s). Given the similarity between adjacent frames, it is feasible to expedite dynamic reconstruction by explicitly fine-tuning the feature vectors.
• Figure 4: Qualitative and quantitative comparison of SOTA methods on the synthetic dataset. Five biological samples arranged from top to bottom are mouse neurons, immune cells, drosophila embryo, microglia in mice and microglia in mice after traumatic brain injury (TBI). Note that due to the presence of large areas without samples in fluorescent microscopy images, the computed PSNR values are often higher than those for natural images.
• Figure 5: Qualitative comparisons of 3DGS and our method on USAF resolution test chart. As shown in the highlighted regions, FFT Loss helps the 3DGS reconstruct more high-frequency details, but the overall quality is still inferior to ours.