Exploring Quasi-Global Solutions to Compound Lens Based Computational Imaging Systems

TL;DR

QGSO tackles the challenge of automatically designing compound-lens computational imaging systems by decoupling initial design generation from end-to-end optimization. It combines OptiFusion, a global-search module using simulated annealing, genetic selection, and ADAM-based local refinement, with EPJO, a memory-efficient, physics-aware joint-optimization pipeline that differentiates through a differentiable PSF model and ISP chain. Experimental results show OptiFusion discovers more diverse, manufacturable starting designs than LensNet, and that end-to-end optimization with EPJO yields measurable improvements in PSNR, SSIM, and LPIPS over traditional separate or CODE V-assisted approaches, especially in extended depth-of-field three-element lenses. The framework thus enables robust global search and effective optics-processor co-design, advancing automated, manufacturable, high-performance computational-imaging systems, with code to be released publicly.

Abstract

Recently, joint design approaches that simultaneously optimize optical systems and downstream algorithms through data-driven learning have demonstrated superior performance over traditional separate design approaches. However, current joint design approaches heavily rely on the manual identification of initial lenses, posing challenges and limitations, particularly for compound lens systems with multiple potential starting points. In this work, we present Quasi-Global Search Optics (QGSO) to automatically design compound lens based computational imaging systems through two parts: (i) Fused Optimization Method for Automatic Optical Design (OptiFusion), which searches for diverse initial optical systems under certain design specifications; and (ii) Efficient Physic-aware Joint Optimization (EPJO), which conducts parallel joint optimization of initial optical systems and image reconstruction networks with the consideration of physical constraints, culminating in the selection of the optimal solution in all search results. Extensive experimental results illustrate that QGSO serves as a transformative end-to-end lens design paradigm for superior global search ability, which automatically provides compound lens based computational imaging systems with higher imaging quality compared to existing paradigms. The source code will be made publicly available at https://github.com/LiGpy/QGSO.

Figures (9)

• Figure 1: Comparison of the design modes for computational imaging systems. (a) shows the separate, sequential design mode. (b) shows the joint design mode that requires manual determination of the initial optical systems. (c) shows the proposed QGSO paradigm (joint design mode in which the algorithm automatically provides the initial optical systems).
• Figure 2: Overview of our compound lens based computational imaging systems design method. Quasi-Global Search Optics (QGSO) includes the Fused Optimization Method for Automatic Optical Design (OptiFusion) and the Efficient Physic-aware Joint Optimization (EPJO). OptiFusion fuses Simulated Annealing (SA), Genetic Algorithm (GA), and ADAM to automatically search for initial structures with sufficient diversity based on traditional optical design metrics. EPJO includes an enhanced differentiable simulation model that incorporates differentiable ray tracing, patch-wise convolution, and an Image Signal Processing (ISP) pipeline. Additionally, EPJO incorporates customized memory-efficient techniques that enable parallel joint optimization of the initial structures discovered by OptiFusion and image reconstruction models, within reasonable computational resources. This approach allows us to select the jointly optimal solution in all search results based on the final reconstructed image quality metrics.
• Figure 3: Comparison between LensNet and OptiFusion in multiple design forms. The same as cote2021deep, each design form is named after their sequence of Glass elements, Air gaps and aperture Stop. The corresponding $\mathcal{L}_{OF}$ is marked above each optical system. LensNet may output the result with overlapping surfaces in some design forms (GASGAGGA, GAGAGASAGA, and GAGGGSAGGA), which makes $\mathcal{L}_{OF}$ abnormally large. The overlapping lens surfaces and the corresponding abnormally large $\mathcal{L}_{OF}$ are marked in red.
• Figure 4: Catalog glasses that meet the design specifications and are available in stock all year round from the Chengdu Guangming Optoelectronic Corporation in China.
• Figure 5: Comparison between CAJD (CODE V Assisted Joint Design), SD (separate design), and QGSO under 3E-I. (a) the initial structures and corresponding final reconstructed image quality of three methods. (b) the resolution chart (ISO 12233) taken by iPhone 12 and zoomed patches were used to evaluate image quality. (c) for each method and from left to right, we show 1) PSFs and corresponding RMS size ($mm$) across $3$Depths (top: D${=}100m$; middle: D${=}10m$; bottom: D${=}5m$) and $3$ HFOV (left: $0^\circ$; middle: $14^\circ$; right: $20^\circ$); 2) degraded zoomed patches (top: D${=}100m$; middle: D${=}10m$; bottom: D${=}5m$); and 3) reconstructed zoomed patches (top: D${=}100m$; middle: D${=}10m$; bottom: D${=}5m$).