OmniLens: Towards Universal Lens Aberration Correction via LensLib-to-Specific Domain Adaptation

TL;DR

OmniLens treats universal CAC as LensLib-to-specific domain adaptation: pre-train on a generated LensLib with CAC loss $L_{CAC}$ and later adapt with a Dark Channel Prior regularization $L_{DCP}$, optimizing $L_{DA} = \lambda_S L_{CAC} + \lambda_T L_{DCP}$ on a small set of real-lens images. The approach yields a diverse, physically plausible LensLib via Evolution-based Automatic Optical Design (EAOD) and a fast unsupervised DA pipeline, achieving strong generalization to unseen lenses and improving lens-specific CAC when descriptions are available. Extensive experiments across multiple networks and real-world data demonstrate PSNR/LPIPS gains and confirm that the pre-trained foundation facilitates few-shot or full training for target lenses. The work provides a practical route to universal CAC with minimal real data, enabling robust corrections for low-end mobile and wearable imaging.

Abstract

Emerging universal Computational Aberration Correction (CAC) paradigms provide an inspiring solution to light-weight and high-quality imaging with a universal model trained on a lens library (LensLib) to address arbitrary lens aberrations blindly. However, the limited coverage of existing LensLibs leads to poor generalization of the trained models to unseen lenses, whose fine-tuning pipeline is also confined to the lens-descriptions-known case. In this work, we introduce OmniLens, a flexible solution to universal CAC via (i) establishing a convincing LensLib with comprehensive coverage for pre-training a robust base model, and (ii) adapting the model to any specific lens designs with unknown lens descriptions via fast LensLib-to-specific domain adaptation. To achieve these, an Evolution-based Automatic Optical Design (EAOD) pipeline is proposed to generate a rich variety of lens samples with realistic aberration behaviors. Then, we design an unsupervised regularization term for efficient domain adaptation on a few easily accessible real-captured images based on the statistical observation of dark channel priors in degradation induced by lens aberrations. Extensive experiments demonstrate that the LensLib generated by EAOD effectively develops a universal CAC model with strong generalization capabilities, which can also improve the non-blind lens-specific methods by 0.35-1.81dB in PSNR. Additionally, the proposed domain adaptation method significantly improves the base model, especially in severe aberration cases (at most 2.59dB in PSNR). The code and data will be available at https://github.com/zju-jiangqi/OmniLens.

Figures (12)

• Figure 1: OmniLens is a flexible solution to universal CAC. In real-world cases of unknown lens descriptions for the applied low-end lenses, the pre-trained model with manually collected LensLib li2021universal delivers poor performance and can hardly be further fine-tuned. In contrast, the automatically generated LensLib in OmniLens develops a more robust model, where the LensLib-to-specific domain adaptation can further enhance the results.
• Figure 2: Overview of the established OmniLens framework. (a): The generation of AODLib by the proposed EAOD method with multiple design specifications and constraints fed in. (b) Based on the AODLib data pairs prepared by imaging simulation, we pre-train a universal CAC model for zero-shot or few-shot CAC of target lenses. (c): Incorporating the real-world captured images of the target lens-description unknown lens, LensLib-to-specific domain adaptation is conducted to fine-tune the pre-trained model for enhancing the results, where the DA training is guided by an unsupervised loss derived from DCP of the optical degradation.
• Figure 3: Overall pipeline of the proposed EAOD method. Taking the lens specifications of $6$ piece as an example, the EAOD leverages a hybrid global and local optimization strategy to seek diverse lens structures that maximally satisfy both imaging quality and physical constraints from generation to generation based on an evolution framework with mutation mechanism.
• Figure 4: Illustration of dark channel prior in CAC. (a): The grayscale histogram of the DC images for aberration-clear image pairs. (b): The corresponding visual samples.
• Figure 5: Visual comparison with different LensLibs under $3$ representative test lenses, which cover different surface types and levels of aberrations. The optical paths of the $3$ lenses are shown on the top of the figure. For each LensLib solution, we present its visual CAC results on an ISO12233 chart produced by FeMaSR which delivers the best perceptual performance.