Disentangle Nighttime Lens Flares: Self-supervised Generation-based Lens Flare Removal

TL;DR

This work addresses the challenge of removing co-occurring nighttime lens flares, including glow and reflective (ghost) flares originating from the same light source. It introduces a physical Nighttime Lens Flare Formation model and a self-supervised two-stream network, SGLFR-Net, comprising PSFR-Net for glow via a PSF rendering prior and TPRR-Net for reflective flare via a texture-prior framework, all guided by an Optical Symmetry Prior. The method achieves state-of-the-art performance on joint flare removal and competitive results on individual glow and reflective flare tasks without any training data, demonstrating strong generalization to real-world images. The approach offers a practical, data-free solution for robust nighttime imaging, with potential benefits for downstream视觉 tasks and semantic understanding under flare conditions.

Abstract

Lens flares arise from light reflection and refraction within sensor arrays, whose diverse types include glow, veiling glare, reflective flare and so on. Existing methods are specialized for one specific type only, and overlook the simultaneous occurrence of multiple typed lens flares, which is common in the real-world, e.g. coexistence of glow and displacement reflections from the same light source. These co-occurring lens flares cannot be effectively resolved by the simple combination of individual flare removal methods, since these coexisting flares originates from the same light source and are generated simultaneously within the same sensor array, exhibit a complex interdependence rather than simple additive relation. To model this interdependent flare relationship, our Nighttime Lens Flare Formation model is the first attempt to learn the intrinsic physical relationship between flares on the imaging plane. Building on this physical model, we introduce a solution to this joint flare removal task named Self-supervised Generation-based Lens Flare Removal Network (SGLFR-Net), which is self-supervised without pre-training. Specifically, the nighttime glow is detangled in PSF Rendering Network(PSFR-Net) based on PSF Rendering Prior, while the reflective flare is modelled in Texture Prior Based Reflection Flare Removal Network (TPRR-Net). Empirical evaluations demonstrate the effectiveness of the proposed method in both joint and individual glare removal tasks.

Figures (8)

• Figure 1: Up to down: (1) Coexisting flared input and its segmentation GT. (2) Deglowed via Flare7K++ 20 and deghosted via RFC 19 results with its segmented version via Wu_2021_CVPR. (3) Deghosted and deglowed results with its segmented version. (4) Result and segmented result of our approach.
• Figure 2: (a) Optical path illustration from the light source to the imaging plane, where the blue line is the ideal light path. The orange line $s$ is the path of scattered light that produces glow flare $R_{s}$, and the red is the path of light refraction between lenses that produce the reflective flare (ghost) $R_{r}$, and $l$ is the line of incident light. (b) Example of lens glare.
• Figure 3: The proposed Self-supervised Generation-based Lens Flare Removal Network (SGLFR-Net) without pre-training is composed with PSF rendering network (PSFR-Net) and texture prior based reflection removal network (TPRR-Net), restricted with Optical Symmetry Based Texture Prior Module (OS-TPM). Two derived priors, namely PSF rendering prior and optical symmetry texture prior are incorporated into our SGLFR-Net on the basis of the Nighttime Lens Flare Formation model.
• Figure 4: Our Optical symmetry-based texture prior(OS-TPM) illustration. Light source mask $M_{s}$ is spatially related to $M_{r}$, obtained via OS-TPM module. Our $M_{r}$ completely covers the manually labelled GT $M_{r}$.
• Figure 5: Joint glow and reflective flare removal Task. Visual comparison of our SGLFR-Net and glow and reflection flare removal SOTA combinations in OurSynDatasets and real world datasets. Notation: (method1)$\rightarrow$(method2).