Dense Cross-Scale Image Alignment With Fully Spatial Correlation and Just Noticeable Difference Guidance

TL;DR

The paper tackles unsupervised dense image alignment under large parallax by introducing a dense cross-scale regression framework that leverages cross-scale feature correlations, a fully spatial correlation module to preserve bidirectional spatial information with low computational cost, and Just Noticeable Distortion (JND) guidance to prioritize perceptually sensitive regions. The method enables a flexible accuracy–efficiency trade-off via the number of scales, and demonstrates state-of-the-art performance on the UDIS-D dataset with notable PSNR/SSIM gains and competitive runtimes. Thorough ablations validate the contributions of cross-scale information, the fully spatial correlation design, and JND guidance, while cross-dataset validation confirms reasonable generalization. Overall, the approach advances unsupervised image alignment by combining scale-aware matching, efficient spatial correlations, and perceptual priors to achieve robust, efficient alignment suitable for real-world applications.

Abstract

Existing unsupervised image alignment methods exhibit limited accuracy and high computational complexity. To address these challenges, we propose a dense cross-scale image alignment model. It takes into account the correlations between cross-scale features to decrease the alignment difficulty. Our model supports flexible trade-offs between accuracy and efficiency by adjusting the number of scales utilized. Additionally, we introduce a fully spatial correlation module to further improve accuracy while maintaining low computational costs. We incorporate the just noticeable difference to encourage our model to focus on image regions more sensitive to distortions, eliminating noticeable alignment errors. Extensive quantitative and qualitative experiments demonstrate that our method surpasses state-of-the-art approaches.

Figures (6)

• Figure 1: Illustration of the proposed alignment network. (a) The overall framework. (b) The proposed cross-scale regression. (c) The proposed fully spatial correlation. Our method uses a coarse-to-fine strategy to estimate the offsets of a mesh. The dense cross-scale regression module integrates the cross-scale information into the local offsets. The fully spatial correlation module utilizes the spatial information of both input features. The dot product is performed between the dark blue and dark red feature vectors to obtain the element in dark gray.
• Figure 2: Illustration of the proposed JND guidance. The JND map is estimated for the reference image. The difference is calculated for the overlapping area to compare with the JND map to update network parameters.
• Figure 3: Ablation studies on our method. The red and green boxes zoom in on the regions with alignment errors.
• Figure 4: Visual comparisons of different alignment methods on the UDIS-D dataset nie2021unsupervised. The colorful boxes zoom in on the area with alignment errors.
• Figure 5: Validation results on the cross-dataset image "19" liao2020single. The red box zooms in on the area with alignment errors.