Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Let There Be Light: Robust Lensless Imaging Under External Illumination With Deep Learning

Eric Bezzam, Stefan Peters, Martin Vetterli

TL;DR

Let There Be Light tackles the practical challenge of external illumination in lensless imaging by integrating an external-illumination estimate into a physics-based, learnable reconstruction pipeline. The authors analyze how forward-model mismatch and ambient light amplify errors, and propose three strategies—direction subtraction, learned subtraction, and concatenation of illumination estimates—to mitigate these effects. They validate the approach on a 25K-measurement dataset collected under varied lighting and demonstrate consistent improvements over baselines, with the best results achieved by combining a pre-processor with the proposed subtraction/concatenation methods. The work enhances the practicality of lensless cameras for fixed installations by sustaining image quality across diverse lighting conditions and provides open-source code and data for broader reuse.

Abstract

Lensless cameras relax the design constraints of traditional cameras by shifting image formation from analog optics to digital post-processing. While new camera designs and applications can be enabled, lensless imaging is very sensitive to unwanted interference (other sources, noise, etc.). In this work, we address a prevalent noise source that has not been studied for lensless imaging: external illumination e.g. from ambient and direct lighting. Being robust to a variety of lighting conditions would increase the practicality and adoption of lensless imaging. To this end, we propose multiple recovery approaches that account for external illumination by incorporating its estimate into the image recovery process. At the core is a physics-based reconstruction that combines learnable image recovery and denoisers, all of whose parameters are trained using experimentally gathered data. Compared to standard reconstruction methods, our approach yields significant qualitative and quantitative improvements. We open-source our implementations and a 25K dataset of measurements under multiple lighting conditions.

Let There Be Light: Robust Lensless Imaging Under External Illumination With Deep Learning

TL;DR

Let There Be Light tackles the practical challenge of external illumination in lensless imaging by integrating an external-illumination estimate into a physics-based, learnable reconstruction pipeline. The authors analyze how forward-model mismatch and ambient light amplify errors, and propose three strategies—direction subtraction, learned subtraction, and concatenation of illumination estimates—to mitigate these effects. They validate the approach on a 25K-measurement dataset collected under varied lighting and demonstrate consistent improvements over baselines, with the best results achieved by combining a pre-processor with the proposed subtraction/concatenation methods. The work enhances the practicality of lensless cameras for fixed installations by sustaining image quality across diverse lighting conditions and provides open-source code and data for broader reuse.

Abstract

Lensless cameras relax the design constraints of traditional cameras by shifting image formation from analog optics to digital post-processing. While new camera designs and applications can be enabled, lensless imaging is very sensitive to unwanted interference (other sources, noise, etc.). In this work, we address a prevalent noise source that has not been studied for lensless imaging: external illumination e.g. from ambient and direct lighting. Being robust to a variety of lighting conditions would increase the practicality and adoption of lensless imaging. To this end, we propose multiple recovery approaches that account for external illumination by incorporating its estimate into the image recovery process. At the core is a physics-based reconstruction that combines learnable image recovery and denoisers, all of whose parameters are trained using experimentally gathered data. Compared to standard reconstruction methods, our approach yields significant qualitative and quantitative improvements. We open-source our implementations and a 25K dataset of measurements under multiple lighting conditions.
Paper Structure (15 sections, 9 equations, 6 figures, 2 tables)

This paper contains 15 sections, 9 equations, 6 figures, 2 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Modeling of Lensless Imaging
  4. Sensitivity to Model Mismatch
  5. Robust Lensless Imaging
  6. Methodology
  7. Direction Subtraction
  8. Learned Subtraction
  9. Concatenate External Illumination
  10. Experiments
  11. Hardware Setup
  12. Reconstruction Approaches
  13. Training and Evaluation
  14. Results
  15. Conclusion

Figures (6)

  • Figure 1: Lensless imaging under external illumination. Recovery is done with the alternating direction method of multipliers (ADMM) method ADMM. Without removing the external illumination through digital post-processing, recovery of a target object may not be discernible (bottom right).
  • Figure 2: Proposed architecture to address external illumination. Alternatively, the external illumination can be concatenated to the raw measurement to input both to the pre-processor.
  • Figure 3: Example setup for measurements under one configuration of external illumination, i.e. ceiling lighting and lamps.
  • Figure 4: Mask and point spread function of the lensless camera prototype used in this work, i.e. multi-focal mask pattern.
  • Figure 5: Example reconstruction outputs from the test set of our dataset collected under varied external illumination. Left-most column shows raw measurement with the external illumination in the inset, while the ground-truth is in the second column.
  • ...and 1 more figures