Towards Lensless Image Deblurring with Prior-Embedded Implicit Neural Representations in the Low-Data Regime

TL;DR

This paper is the first to leverage implicit neural representations for lensless image deblurring, achieving reconstructions without the requirement of prior training, effectively bridging the gap between the no-data and high-data regimes.

Abstract

The field of computational imaging has witnessed a promising paradigm shift with the emergence of untrained neural networks, offering novel solutions to inverse computational imaging problems. While existing techniques have demonstrated impressive results, they often operate either in the high-data regime, leveraging Generative Adversarial Networks (GANs) as image priors, or through untrained iterative reconstruction in a data-agnostic manner. This paper delves into lensless image reconstruction, a subset of computational imaging that replaces traditional lenses with computation, enabling the development of ultra-thin and lightweight imaging systems. To the best of our knowledge, we are the first to leverage implicit neural representations for lensless image deblurring, achieving reconstructions without the requirement of prior training. We perform prior-embedded untrained iterative optimization to enhance reconstruction performance and speed up convergence, effectively bridging the gap between the no-data and high-data regimes. Through a thorough comparative analysis encompassing various untrained and low-shot methods, including under-parameterized non-convolutional methods and domain-restricted low-shot methods, we showcase the superior performance of our approach by a significant margin.

Figures (6)

• Figure 1: The complete untrained iterative optimization procedure using the implicit neural representation framework has been illustrated here. We coordinate input $(u, v)$ for the MLP $\mathcal{M}_\theta$. Intermediate reconstruction $\mathcal{M}_\theta(u, v)$, which is then convolved with the known point spread function (PSF) $k$ to produce the intermediate lensless image $k \ast \mathcal{M}_\theta(u, v)$. $L_{MSE}$ between this intermediate lensless image and the original lensless image $y$ is backpropagated to update $\mathcal{M}_\theta$.
• Figure 2: A. Prior-Embedding process: A domain-restricted image is represented using the INR simply by optimizing the MSE between the network output and the image. B. Prior-Embedded Physics-informed Inverse Imaging: The prior embedded INR $\mathcal{M}_{prior}$ is the starting point of the physics-informed optimization process that requires the PSF.
• Figure 3: PSNR versus iteration plots. The left plot depicts the results from the Deep Decoder framework, while the right plot shows the outcomes of our method. The parameter details have been highlighted below each plot.
• Figure 4: Visual comparison of reconstruction results of existing methods like the ADMMantipa2018diffusercam in the extreme left column, followed by Le-ADMM-UNet monakhova2019learned, Deep Decoder heckel_deep_2018 modified for lensless image deblurring by banerjee2023physics, against our untrained INR. The quantitative scores, presented in PSNR (in dB) | SSIM format, are provided below each image.
• Figure 5: Visual comparison of reconstruction results of our untrained INR against a modified Deep Decoder. The parameter count of the network and the quantitative scores, presented in Parameter Count | PSNR (in dB) | SSIM format, are provided below each image.