KeDuSR: Real-World Dual-Lens Super-Resolution via Kernel-Free Matching

TL;DR

KeDuSR tackles real-world dual-lens SR by aligning the telephoto reference to the LR image center through a center warping pipeline and by performing kernel-free matching between LR-corner and LR-center to warp the reference to the corners. A decoupled SISR encoder and an adaptive fusion module jointly leverage the LR content and the warped reference to produce high-quality HR outputs, while a well-curated DuSR-Real dataset enables supervised learning on real captures. Empirical results across three real-world datasets show consistent improvements over state-of-the-art RefSR and dual-lens SR methods, with strong generalization to unseen data. The work also provides a valuable real-world DuSR dataset resource and empirical evidence that center-aligned warping plus kernel-free corner matching enhances both center and corner reconstruction, reducing artifacts and improving detail fidelity.

Abstract

Dual-lens super-resolution (SR) is a practical scenario for reference (Ref) based SR by utilizing the telephoto image (Ref) to assist the super-resolution of the low-resolution wide-angle image (LR input). Different from general RefSR, the Ref in dual-lens SR only covers the overlapped field of view (FoV) area. However, current dual-lens SR methods rarely utilize these specific characteristics and directly perform dense matching between the LR input and Ref. Due to the resolution gap between LR and Ref, the matching may miss the best-matched candidate and destroy the consistent structures in the overlapped FoV area. Different from them, we propose to first align the Ref with the center region (namely the overlapped FoV area) of the LR input by combining global warping and local warping to make the aligned Ref be sharp and consistent. Then, we formulate the aligned Ref and LR center as value-key pairs, and the corner region of the LR is formulated as queries. In this way, we propose a kernel-free matching strategy by matching between the LR-corner (query) and LR-center (key) regions, and the corresponding aligned Ref (value) can be warped to the corner region of the target. Our kernel-free matching strategy avoids the resolution gap between LR and Ref, which makes our network have better generalization ability. In addition, we construct a DuSR-Real dataset with (LR, Ref, HR) triples, where the LR and HR are well aligned. Experiments on three datasets demonstrate that our method outperforms the second-best method by a large margin. Our code and dataset are available at https://github.com/ZifanCui/KeDuSR.

Figures (10)

• Figure 1: Illustration of our DuSR-Real dataset construction process and the resolution gap between LR and Ref.
• Figure 2: Illustration of the similarity between the corner and center regions, where (a) shows the similar patch pairs and (b) is the matching curve. The LR-center region is circled by a white dotted box and $\bar{I}^\text{Ref}$ is its corresponding HR Ref.
• Figure 3: The framework of our KeDuSR. Kernel-Free Matching is performed between LR-corner and LR-center($I^\text{LRC}$) to obtain the index map $M$ and confidence map $C$. Then, employing center warping and corner warping, we obtain the warped high-resolution feature map $\tilde{F}^\text{Ref}$ of the reference. After fusion with $F^\text{LR}$, we generate the SR result $I^\text{SR}$.
• Figure 4: Visual comparisons on real-world dual-lens datasets. The white dotted box indicates the overlapped FoV area between LR and Ref. The presented results are obtained with only reconstruction loss.
• Figure 5: Exemplars of the aligned HR images in our DuSR-Real dataset.