LensNet: An End-to-End Learning Framework for Empirical Point Spread Function Modeling and Lensless Imaging Reconstruction

TL;DR

LensNet addresses the lensless imaging challenge by learning a forward model $I_{ ext{measurement}} = I_{ ext{object}} * \operatorname{PSF} + \text{noise}$ and reconstructing $I_{ ext{object}}$ through an end-to-end encoder–decoder that fuses spatial-domain and frequency-domain cues. A learnable Coded Mask Simulator models the PSF dynamics, while Wiener-filter-based frequency-domain refinements enhance denoising and edge fidelity. The approach achieves state-of-the-art PSNR/SSIM/LPIPS on DiffuserCam and MWDNs, with qualitative results showing sharper edges and preserved textures. This framework enables robust, ultra-compact imaging suitable for miniature sensors and medical diagnostics by combining physics-based modeling with data-driven learning.

Abstract

Lensless imaging stands out as a promising alternative to conventional lens-based systems, particularly in scenarios demanding ultracompact form factors and cost-effective architectures. However, such systems are fundamentally governed by the Point Spread Function (PSF), which dictates how a point source contributes to the final captured signal. Traditional lensless techniques often require explicit calibrations and extensive pre-processing, relying on static or approximate PSF models. These rigid strategies can result in limited adaptability to real-world challenges, including noise, system imperfections, and dynamic scene variations, thus impeding high-fidelity reconstruction. In this paper, we propose LensNet, an end-to-end deep learning framework that integrates spatial-domain and frequency-domain representations in a unified pipeline. Central to our approach is a learnable Coded Mask Simulator (CMS) that enables dynamic, data-driven estimation of the PSF during training, effectively mitigating the shortcomings of fixed or sparsely calibrated kernels. By embedding a Wiener filtering component, LensNet refines global structure and restores fine-scale details, thus alleviating the dependency on multiple handcrafted pre-processing steps. Extensive experiments demonstrate LensNet's robust performance and superior reconstruction quality compared to state-of-the-art methods, particularly in preserving high-frequency details and attenuating noise. The proposed framework establishes a novel convergence between physics-based modeling and data-driven learning, paving the way for more accurate, flexible, and practical lensless imaging solutions for applications ranging from miniature sensors to medical diagnostics. The link of code is https://github.com/baijiesong/Lensnet.

Figures (11)

• Figure 1: Performance comparison on the MWDNs dataset and the DiffuserCam dataset in terms of PSNR and SSIM.
• Figure 2: Visualized results on the DiffuserCam dataset. Each image shows the input image, the LensNet reconstruction result, and the corresponding ground truth.
• Figure 3: The figure illustrates the detailed structure of our model. The input image is measurement on the system and the result is clear scene image through our framework.
• Figure 4: The above shows the qualitative experimental results on MWDNs Dataset. Our method effectively achieves high-quality reconstruction, comparing other methods.
• Figure 5: This figure presents the results of our ablation study, highlighting the impact of various components on model performance.