End-to-End Hybrid Refractive-Diffractive Lens Design with Differentiable Ray-Wave Model

TL;DR

This work tackles the challenge of accurately simulating and optimizing hybrid refractive-diffractive lenses for high-quality imaging. It introduces a differentiable ray-wave imaging framework that fuses coherent ray tracing with wave diffraction and places the DOE between the final surface and the sensor, enabling end-to-end optimization of optics and reconstruction networks. The authors validate the approach by comparing PSFs to scalar diffraction ground truth and Zemax results, showing higher fidelity, and demonstrate a real hybrid aspherical-DOE prototype achieving aberration correction and wide-field EDoF imaging. The framework enables accurate modeling of discontinuous DOE phase maps and large-FoV systems, with potential to advance computational imaging and practical camera design.

Abstract

Hybrid refractive-diffractive lenses combine the light efficiency of refractive lenses with the information encoding power of diffractive optical elements (DOE), showing great potential as the next generation of imaging systems. However, accurately simulating such hybrid designs is generally difficult, and in particular, there are no existing differentiable image formation models for hybrid lenses with sufficient accuracy. In this work, we propose a new hybrid ray-tracing and wave-propagation (ray-wave) model for accurate simulation of both optical aberrations and diffractive phase modulation, where the DOE is placed between the last refractive surface and the image sensor, i.e. away from the Fourier plane that is often used as a DOE position. The proposed ray-wave model is fully differentiable, enabling gradient back-propagation for end-to-end co-design of refractive-diffractive lens optimization and the image reconstruction network. We validate the accuracy of the proposed model by comparing the simulated point spread functions (PSFs) with theoretical results, as well as simulation experiments that show our model to be more accurate than solutions implemented in commercial software packages like Zemax. We demonstrate the effectiveness of the proposed model through real-world experiments and show significant improvements in both aberration correction and extended depth-of-field (EDoF) imaging. We believe the proposed model will motivate further investigation into a wide range of applications in computational imaging, computational photography, and advanced optical design. Code will be released upon publication.

Figures (8)

• Figure 1: We propose a differentiable ray-wave imaging model that accurately simulates both aberrations and phase modulation while enabling end-to-end optimization for the hybrid refractive-diffractive lens and the image reconstruction network. To experimentally demonstrate the effectiveness of the proposed model, we present a hybrid aspherical-DOE lens prototype (left) and investigate two applications: aberration correction (middle) and large field-of-view extended depth-of-field imaging (right).
• Figure 2: A differentiable ray-wave model is proposed, enabling accurate simulation of both refractive aberrations and diffractive phase modulations. The ray-wave model initially calculates the complex wave field at the DOE plane using coherent ray-tracing. The aberrated wave field encodes the amplitude and phase aberrations introduced by the refractive lens. Subsequently, the wave field is modulated by the DOE phase profile and propagated to the sensor image plane for PSF calculation. The sensor-captured images are simulated with full FoV RGB PSFs and then fed into the downstream image reconstruction network. The pipeline is fully differentiable, enabling gradient backpropagation for end-to-end design of both the optics and the neural network.
• Figure 3: PSF simulation of different optical models. An ideal paraxial thin lens with a DOE is used for testing. For this specific case, wave optics gives accurate simulation results goodman2005introduction, therefore serving as the ground truth. Different DOE phase maps are studied, our ray-wave model gives identical results with the ground truth, while the ray tracing model fails to address the real diffractive phenomena and can not function for discontinuous phase maps.
• Figure 4: Comparison of image quality between different hybrid lens design approaches. The paraxial wave optics model fails to correct aberrations across the full FoV, resulting in degraded image quality in both raw simulations and network reconstructions. The ray tracing method in Zemax corrects aberrations but uses the RMS spot size as the objective, precluding end-to-end lens design. Our ray-wave model accurately simulates full-FoV PSFs and is fully differentiable, enabling end-to-end co-design of the hybrid lens and the network, thus achieving the best reconstruction quality. Raw simulated images and corresponding reconstructed images are shown for each design approach, with zoomed-in patches highlighting the differences in image quality across the FoV.
• Figure 5: Hybrid lens designs using different optical models and optimization methods. Top: achromatic DOE design using the paraxial optical model, minimizing paraxial chromatic aberration. Middle: hybrid lens design using a ray-tracing model, optimizing the RMS spot size in Zemax. Bottom: hybrid lens design using the proposed ray-wave model, optimizing the final image quality. PSFs at different FoVs are calculated with the proposed ray-wave model, shown in the log scale.