Designing lensless imaging systems to maximize information capture

TL;DR

This paper addresses how to maximize information capture in mask-based lensless imaging by relating object sparsity, encoder multiplexing, and noise through mutual information. It introduces a decoder-independent, measurement-based evaluation framework that uses $I(\mathbf{O}; \mathbf{Y}) = I(\mathbf{X}; \mathbf{Y})$ and estimates $H(\mathbf{Y})$ and $H(\mathbf{Y}|\mathbf{X})$ to compare encoders with varying multiplexing. It then develops Information-Driven Encoder Analysis Learning (IDEAL) to design information-optimal encoders for fixed object distributions, and validates these ideas experimentally, showing that dense objects benefit from less multiplexing while sparse objects benefit from more, with a constant sensor sparsity across optimal encoders. Overall, the work provides principled, object-distribution-specific design guidelines and a framework applicable to general multiplexing systems.

Abstract

Mask-based lensless imaging uses an optical encoder (e.g. a phase or amplitude mask) to capture measurements, then a computational decoding algorithm to reconstruct images. In this work, we evaluate and design lensless encoders based on the information content of their measurements using mutual information estimation. Our approach formalizes the object-dependent nature of lensless imaging and quantifies the interdependence between object sparsity, encoder multiplexing, and noise. Our analysis reveals that optimal encoder designs should tailor encoder multiplexing to object sparsity for maximum information capture, and that all optimally-encoded measurements share the same level of sparsity. Using mutual information-based optimization, we design information-optimal encoders for compressive imaging of fixed object distributions. Our designs demonstrate improved downstream reconstruction performance for objects in the distribution, without requiring joint optimization with a specific reconstruction algorithm. We validate our approach experimentally by evaluating lensless imaging systems directly from captured measurements, without the need for image formation models, reconstruction algorithms, or ground truth data. Our comprehensive analysis establishes design and engineering principles for lensless imaging systems, and offers a model for the study of general multiplexing systems, especially those with object-dependent performance.

Figures (12)

• Figure 1: Mutual information estimation framework for lensless imaging. a) A phase mask encoder maps an object distribution $\mathbf{O}$ to a noiseless image distribution $\mathbf{X}$. Image detection adds noise, forming a distribution of noisy sensor measurements $\mathbf{Y}$. Mutual information, estimated from the noisy sensor measurement distribution, serves as a metric for analyzing and designing the encoder. b) The lensless image formation model takes each object $\mathbf{o}$ and encodes it into a noiseless image $\mathbf{x}$ by convolving with the encoder point spread function $\mathbf{h}$. c) To analyze the effects of multiplexing in lensless imaging, we study encoders with one through nine lenslets, corresponding to increasing multiplexing levels.
• Figure 2: Mutual information for varying object sparsity and encoder multiplexing. a) Examples of simulated objects with increasing levels of sparsity as quantified by the Tamura coefficient. b) Mutual information for one through nine lenslet multiplexing systems, each swept across object sparsity levels. As multiplexing increases, the maximum mutual information (denoted by stars) is achieved at higher sparsity levels. c) Maximum mutual information (denoted by stars) corresponds to a fixed sensor sparsity across all multiplexing encoders. d) Example measurements corresponding to optimal fixed sensor sparsity qualitatively differ from examples with low or high sparsity.
• Figure 3: Information-optimal phase mask design for lenslet-based encoders with Information-Driven Encoder Analysis Learning (IDEAL). a) Lensless imaging enables compressive extended field-of-view imaging in 2D by encoding an object larger than the finite extent of an image sensor into a limited measurement region through multiplexing. A decoder recovers an extended field-of-view from the measurement. b) Examples of objects with varying sparsity. c) Visualization of the information-optimal encoder, point spread function, and example measurements for each object distribution. As the object sparsity increases, the number of lenslets (multiplexing level) in the corresponding information-optimal design increases. d) Mutual information for IDEAL designs for each object distribution. Random initialization with 25 lenslets is denoted by an "X" and the resulting IDEAL design is denoted by a diamond. Information-optimal designs have higher information than baseline heuristic designs (denoted by circles). 95% confidence intervals are smaller than the marker size for IDEAL designs and indicated by shading for heuristic designs. e) Higher mutual information in IDEAL designs results in improved reconstruction performance for each object distribution, as quantified by reconstruction peak signal-to-noise ratio.
• Figure 4: Mutual information in experimental lensless imaging comparing a traditional lens to two phase mask encoders: a diffuser and a random multi-focal lenslet (RML) array. a) Mutual information is estimated from experimentally-captured measurements of natural images for each imaging system. b) 1D cross-section of the modulation transfer function for each encoder. The high-multiplexing diffuser has the worst frequency transmittance. c) The lens system has the most mutual information and information decreases with multiplexing. d) Examples of reconstructions for the RML and diffuser systems compared to the lens system show worse image quality with the high-multiplexing diffuser. The reconstruction structural similarity index measure (SSIM) for the RML (SSIM = 0.886) is higher than the diffuser (SSIM = 0.821).
• Figure S1: The effect of patch size on estimated mutual information. Larger patch sizes result in lower estimated mutual information, and level off after $10 \times 10$ pixel patch size.