A Physics-Informed Blur Learning Framework for Imaging Systems

TL;DR

This work introduces a physics-informed blur learning framework for imaging systems that learns a wavefront-based PSF model without requiring lens parameters. It handles spatially varying blur by decomposing the wavefront into a directional basis, applying curriculum learning from the image center to the edge, and first estimating monochromatic PSFs before chromatic shifts across color channels. The two-stage PSF estimation is guided by spatial frequency measurements and chromatic-area differences, enabling high-fidelity PSF reconstruction and improved downstream deblurring when used to train state-of-the-art deblurring networks. The approach demonstrates superior PSF accuracy and deblurring quality in simulations and real captures, with potential for broad deployment across photography, microscopy, and automotive imaging while noting unresolved wide-field chromatic aberration corrections.

Abstract

Accurate blur estimation is essential for high-performance imaging across various applications. Blur is typically represented by the point spread function (PSF). In this paper, we propose a physics-informed PSF learning framework for imaging systems, consisting of a simple calibration followed by a learning process. Our framework could achieve both high accuracy and universal applicability. Inspired by the Seidel PSF model for representing spatially varying PSF, we identify its limitations in optimization and introduce a novel wavefront-based PSF model accompanied by an optimization strategy, both reducing optimization complexity and improving estimation accuracy. Moreover, our wavefront-based PSF model is independent of lens parameters, eliminate the need for prior knowledge of the lens. To validate our approach, we compare it with recent PSF estimation methods (Degradation Transfer and Fast Two-step) through a deblurring task, where all the estimated PSFs are used to train state-of-the-art deblurring algorithms. Our approach demonstrates improvements in image quality in simulation and also showcases noticeable visual quality improvements on real captured images.

Figures (11)

• Figure 1: We introduce a point spread function (PSF) estimation framework and demonstrate its effectiveness in deblurring. From left to right: PSF estimated via Degradation Transfer chen2021extreme (state-of-the-art), PSF estimated by our method, and the ground-truth PSF of lens #63762 from Edmund; a blurry input image synthesized using an image from the FiveK dataset fivek and the ground-truth PSF; the patches from the blurry input image; the patches deblurred by pre-trained Restormers zamir2022restormer using training data generated from PSFs obtained through our method and Degradation Transfer chen2021extreme, respectively; and the corresponding ground truth patches. Our approach outperforms existing state-of-the-art method in both PSF estimation accuracy and deblurring quality.
• Figure 2: Diagram of wavefront aberration and PSF. When light passes through an aberrated optical system, the real wavefront deviates from the ideal, causing defocus in the imaging plane. This deviation, varying with incidence angle and wavelength, creates a spatially varying, symmetric PSF. We focus on the PSF along the +Y axis, where normalized field height $\mathrm{H}$ and wavelength $\lambda$ define $\hbox{PSF}(\mathrm{H}, \lambda)$. Other PSFs are generated by rotating $\hbox{PSF}(\mathrm{H}, \lambda)$ by angle $\phi$ from the +Y axis, with positive $\phi$ indicating clockwise rotation (yellow box).
• Figure 3: An example demonstrating how the proposed wavefront basis mitigates gradient conflicts. Top: In the Seidel PSF model, the spherical aberration basis $\rho^2$ creates a circular PSF shape with $360^\circ$ of blur (orange arrow). This produces identical SFR in both the $0^\circ$ and $90^\circ$ directions. When attempting to optimize the coefficient $W_0$ to match real SFR measurements, which differ between $0^\circ$ and $90^\circ$, gradient conflicts arise. Bottom: In our proposed wavefront basis, each basis affects the SFR in only one direction. This allows the model to independently adjust coefficients $W_1$ and $W_2$ to better match the measured SFR without gradient conflict.
• Figure 4: Diagram of the proposed two-step PSF estimation framework, the first step involves learning monochromatic aberration per normalized image height $\mathrm{H}$. The network $\mathcal{G}_{\Theta1}$ processes $\mathrm{H}$ and $\mathrm{H}^2$ to output coefficients, generate wavefront aberration and transform it into the $\hbox{PSF}^*$, followed by calculating the modulation transfer function $\hbox{MTF}^*$, resulting in the spatial frequency response ( $\hbox{SFR}^*$) curve. Concurrently, a real $\hbox{SFR}$ curve at the same $\mathrm{H}$ of one color channel is derived from real capture. Discrepancies between these curves guide $\mathcal{G}_{\Theta1}$ to faithfully represent real aberration. The second step focuses on learning PSF shifts across channels. Using $\mathrm{H}$ as input, $\mathcal{G}_{\Theta2}$ calculates shifts, generates shifted PSF, and produces chromatic areas $\hbox{CA}^*$ through a physical process. Real chromatic areas $\hbox{CA}$ data at the same $\mathrm{H}$ are obtained from captures, the disparities between the two data guiding $\mathcal{G}_{\Theta2}$ to output $\hbox{CA}^*$ faithfully representing reality. These two steps enable the learning of spatial-variant PSF of the whole imaging system.
• Figure 5: Estimated PSFs and ground-truth: The PSFs are arranged from left to right by increasing normalized field height $\mathrm{H}$. From top to bottom, the PSF estimates using Degradation Transfer chen2021extreme, Fast Two-step eboli2022fast, and our method, followed by the ground-truth PSF of lens #63762 from Edmund.