OmniLens++: Blind Lens Aberration Correction via Large LensLib Pre-Training and Latent PSF Representation

TL;DR

OmniLens++ tackles blind lens aberration correction by combining scalable LensLib construction (AODLibpro) with a latent PSF representation (LPR) to inject degradation priors into a foundational CAC model (FoundCAC). The PSF priors are learned via PSF-VQVAE and an Optical Degradation Network, and are fused into a CAC backbone to guide correction. Across real-world and synthetic benchmarks, OmniLens++ achieves state-of-the-art zero-shot generalization, with measurable gains from both expanded data diversity and latent-PSF guidance. The work offers a scalable, reproducible framework for robust aberration correction in diverse optical systems, including minimalist, high-end, and metalens architectures, with public code and data planned.

Abstract

Emerging deep-learning-based lens library pre-training (LensLib-PT) pipeline offers a new avenue for blind lens aberration correction by training a universal neural network, demonstrating strong capability in handling diverse unknown optical degradations. This work proposes the OmniLens++ framework, which resolves two challenges that hinder the generalization ability of existing pipelines: the difficulty of scaling data and the absence of prior guidance characterizing optical degradation. To improve data scalability, we expand the design specifications to increase the degradation diversity of the lens source, and we sample a more uniform distribution by quantifying the spatial-variation patterns and severity of optical degradation. In terms of model design, to leverage the Point Spread Functions (PSFs), which intuitively describe optical degradation, as guidance in a blind paradigm, we propose the Latent PSF Representation (LPR). The VQVAE framework is introduced to learn latent features of LensLib's PSFs, which is assisted by modeling the optical degradation process to constrain the learning of degradation priors. Experiments on diverse aberrations of real-world lenses and synthetic LensLib show that OmniLens++ exhibits state-of-the-art generalization capacity in blind aberration correction. Beyond performance, the AODLibpro is verified as a scalable foundation for more effective training across diverse aberrations, and LPR can further tap the potential of large-scale LensLib. The source code and datasets will be made publicly available at https://github.com/zju-jiangqi/OmniLens2.

Figures (25)

• Figure 1: This work addresses the challenges of the current LensLib-PT pipeline. (a) OmniLens suffers from limited data scalability arising from the biased data distribution and lack of prior guidance for the model. (b) In OmniLens++, the proposed AODLibpro reveals uniform distribution contributing to improved scalability, while the latent PSF representation provides effective prior guidance with blind paradigm. (c) OmniLens++ effectively resolves the failure case of OmniLens.
• Figure 2: Illustration of the key designs in constructing AODLibpro. We expand surface type and imaging distance specifications in (a) to realize a broader set of optical degradation patterns during lens source generation; and quantify degradation severity and spatial variation trends via image quality assessment in (b), yielding a hybrid sampling that covers plausible optical degradation patterns.
• Figure 3: Overview of the proposed FoundCAC model guided by pre-trained LPR. (a) Pre-training stage for learning LPR. PSF-VQVAE explicitly stores the key latent PSF features regularized by ODN for modeling optical priors. (b) Aberration correction stage for training FoundCAC. The latent PSF features are predicted for guiding the correction model, regularized by the learned LPR.
• Figure 4: Visual results of representative blind lens aberration correction pipelines across real-world cases. More results and the capture details are provided in Appendix \ref{['sup:nriqa']} and \ref{['sup:real_visual']}.
• Figure 5: AODLibpro v.s. AODLib-EAOD in terms of uniformity of aberration distributions, coverage over real-world lenses, and scalability. (a) Histogram of degradation type distributions in the sampled lenses. (b) LensLib coverage visualization based on OIQ evaluated per FoV and wavelength. (c) Improvements of SwinIR trained with LensLibs of different scales over the method without LensLib eboli2022fast. The improvement is averaged across the PSNR and LPIPS.