V2V3D: View-to-View Denoised 3D Reconstruction for Light-Field Microscopy

TL;DR

V2V3D tackles noisy, snapshot light-field microscopy by jointly denoising and reconstructing 3D volumes in an unsupervised, view-to-view framework. It splits views into two non-overlapping subsets, trains two branches to predict volumes, and uses forward projection with PSF priors coupled with a wave-optics-based feature alignment to preserve high-frequency details; the final volumes are fused. The method introduces a loss combining MSE, FFT-based frequency preservation, and a de-crosstalk term to suppress cross-view artifacts. Experiments on synthetic and real 2pSAM data show state-of-the-art denoising and reconstruction efficiency, with macro-scale validation and ablation confirming each component's contribution.

Abstract

Light field microscopy (LFM) has gained significant attention due to its ability to capture snapshot-based, large-scale 3D fluorescence images. However, existing LFM reconstruction algorithms are highly sensitive to sensor noise or require hard-to-get ground-truth annotated data for training. To address these challenges, this paper introduces V2V3D, an unsupervised view2view-based framework that establishes a new paradigm for joint optimization of image denoising and 3D reconstruction in a unified architecture. We assume that the LF images are derived from a consistent 3D signal, with the noise in each view being independent. This enables V2V3D to incorporate the principle of noise2noise for effective denoising. To enhance the recovery of high-frequency details, we propose a novel wave-optics-based feature alignment technique, which transforms the point spread function, used for forward propagation in wave optics, into convolution kernels specifically designed for feature alignment. Moreover, we introduce an LFM dataset containing LF images and their corresponding 3D intensity volumes. Extensive experiments demonstrate that our approach achieves high computational efficiency and outperforms the other state-of-the-art methods. These advancements position V2V3D as a promising solution for 3D imaging under challenging conditions.

Figures (7)

• Figure 1: Background and concept of V2V3D. (a) Previous methods, such as VCDNet vcdnet and DINER DINER, directly apply all views for reconstruction and lack physical priors in feature representation. Therefore, when reconstructing real-world noisy scenes, these methods usually generate results with conspicuous artifacts and blurriness. (b) The proposed method divides the noisy views into two non-overlapping subsets and employs two networks to generate the corresponding volumes. Additionally, we incorporate PSF priors for feature alignment, thereby enhancing feature aggregation across views. (c) Through the aforementioned custom designs, our method achieves state-of-the-art performance.
• Figure 2: The overall framework of V2V3D, which divides all views into two subsets, with each subset generating a corresponding volume that collaborates to effectively reduce noise. $\circledast$ denotes the 2D convolution operation. Additionally, V2V3D incorporates a novel wave-optics-based feature alignment technique, leveraging PSF priors to enhance the recovery of high-frequency information.
• Figure 3: The diagram of the proposed wave-optics-based feature alignment module. The features extracted from different views are misaligned in the spatial dimension. To address this, we use kernels generated from the PSFs to align these features, thereby facilitating subsequent feature aggregation.
• Figure 4: Qualitative comparisons on the synthetic dataset. Two biological samples arranged from top to bottom are B cells and vessels. Our solution delivers significantly higher quality, with less noise and sharper details.
• Figure 5: Qualitative comparisons on the synthetic dataset. Two biological samples arranged from top to bottom are microglia and dendrites. Our solution delivers significantly higher quality, with less noise and sharper details.