Computational metaoptics for imaging

TL;DR

This review explores the synergistic integration of metaoptics and computational imaging,"computational metaoptics,"which combines the physical wavefront shaping ability of metasurfaces with advanced computational algorithms to enhance imaging performance beyond conventional limits.

Abstract

Metasurfaces -- ultrathin structures composed of subwavelength optical elements -- have revolutionized light manipulation by enabling precise control over electromagnetic waves' amplitude, phase, polarization, and spectral properties. Concurrently, computational imaging leverages algorithms to reconstruct images from optically processed signals, overcoming limitations of traditional imaging systems. This review explores the synergistic integration of metaoptics and computational imaging, "computational metaoptics," which combines the physical wavefront shaping ability of metasurfaces with advanced computational algorithms to enhance imaging performance beyond conventional limits. We discuss how computational metaoptics addresses the inherent limitations of single-layer metasurfaces in achieving multifunctionality without compromising efficiency. By treating metasurfaces as physical preconditioners and co-designing them with reconstruction algorithms through end-to-end (inverse) design, it is possible to jointly optimize the optical hardware and computational software. This holistic approach allows for the automatic discovery of optimal metasurface designs and reconstruction methods that significantly improve imaging capabilities. Advanced applications enabled by computational metaoptics are highlighted, including phase imaging and quantum state measurement, which benefit from the metasurfaces' ability to manipulate complex light fields and the computational algorithms' capacity to reconstruct high-dimensional information. We also examine performance evaluation challenges, emphasizing the need for new metrics that account for the combined optical and computational nature of these systems. Finally, we identify new frontiers in computational metaoptics which point toward a future where computational metaoptics may play a central role in advancing imaging science and technology.

Figures (5)

• Figure 1: Computational imaging with metaoptics: degrees of freedom, physics, and algorithms.a. The general goal of a computational imaging device is to reconstruct various degrees of freedom of an incident light field, for instance its polarization, frequency, momentum, and complex amplitude distribution. Advanced degrees of freedom (e.g., density matrix of the quantum state of light) may also be of interest. b. Light manipulation is realized by leveraging physical properties of metaoptical devices, such as their ability to locally control the complex amplitude of an incoming wavefront, engineered spectral dispersion, non-locality (spatial dispersion), active control of physical properties (e.g., complex transmission), and nonlinear optical properties. c. Once imaged by a detector, the signal may be reconstructed using various reconstruction and estimation methods, such as least-square error minimization (which may include priors on the reconstructed degrees of freedom, such as high sparsity or low complexity). Black-box methods, such as fully connected neural networks, may also be utilized to classify detected signals and images. Other parameter estimation methods, such as maximum likelihood estimation (MLE) may also be used to estimate the degrees of freedom of the incident light field.
• Figure 2: End-to-end computational metaoptic imaging via synergistic inverse design of geometrical degrees of freedom and hyperparameters.a. Light emanating from a scene or object is processed with a metaoptics (corresponding to the system's "front end") and imaged onto a detector. Additional optical or electronic components (not shown) may be used to further process light or the generated signal on the detector. b. The metaoptical device is defined by its geometry, which can be optimized via surrogate models or free-form (topology) optimization, or a combination thereof. c. The generated signal is processed by an optimizer (or estimator, corresponding to the system's "back end"), generating a reconstructed object. The calculated error signal is back-propagated to adjust reconstruction hyperparameters and metaoptics geometry, in order to reduce the error.
• Figure 3: End-to-end optimized multi-spectral imaging. A metasurface captures a multispectral object (left) and produces a greyscale image (500x500 pixels), from which a computer reconstructs the multispectral ground truth (16 spectral channels, each with 50x50 pixels).
• Figure 4: Advanced applications enabled by computational metaoptic imaging.a. A compact quantitative phase gradient microscope based on 3-step phase shifting using a spatially-multiplexed bi-layer metasurface. Reproduced with permission from Ref. kwon2020single. b. A Fourier optical spin splitting microscope by inserting a polarization-multiplexed metasurface into the Fourier plane of a conventional 4$f$ imaging system. Reproduced with permission from Ref. zhou2022fourier. c. Single-shot deterministic complex amplitude imaging with a single-layer metalens based on polarization phase-shifted shearing. Reproduced with permission from Ref. li2024single. d. A mechanical scanning-free transport-of-intensity-based quantitative phase imaging microscope using a spectrally-dispersive metalens. Reproduced with permission from Ref. wang2024quantitative. e. A mechanical scanning-free transport-of-intensity-based quantitative phase imaging microscope using a polarization-multiplexed metalens. Reproduced with permission from Ref. min2024varifocal. f. A single-shot transport-of-intensity-based quantitative phase imaging microscope using a polarization-multiplexed metasurface. Reproduced with permission from Ref. engay2021polarization.
• Figure 5: Two recent examples of metaoptics for quantum photonic state measurement and reconstruction.a. A metasurface for reconstructing density matrices represented in the orbital angular momenta (OAM) basis. Reproduced with permission from Ref. Wang2023characterization. b. A metasurface composed of a single periodic meta-grating that can perform an optimized set of generalized polarization measurements represented by general POVMs, which can robustly reconstruct polarization density matrices. Reproduced with permission from Ref. Lung2024robust.