A Robust Multisource Remote Sensing Image Matching Method Utilizing Attention and Feature Enhancement Against Noise Interference

TL;DR

A robust multisource RSI matching method utilizing attention and feature enhancement against noise interference and an outlier removal network based on a binary classification mechanism, which can establish effective and geometrically consistent correspondences between images is proposed.

Abstract

Image matching is a fundamental and critical task of multisource remote sensing image applications. However, remote sensing images are susceptible to various noises. Accordingly, how to effectively achieve accurate matching in noise images is a challenging problem. To solve this issue, we propose a robust multisource remote sensing image matching method utilizing attention and feature enhancement against noise interference. In the first stage, we combine deep convolution with the attention mechanism of transformer to perform dense feature extraction, constructing feature descriptors with higher discriminability and robustness. Subsequently, we employ a coarse-to-fine matching strategy to achieve dense matches. In the second stage, we introduce an outlier removal network based on a binary classification mechanism, which can establish effective and geometrically consistent correspondences between images; through weighting for each correspondence, inliers vs. outliers classification are performed, as well as removing outliers from dense matches. Ultimately, we can accomplish more efficient and accurate matches. To validate the performance of the proposed method, we conduct experiments using multisource remote sensing image datasets for comparison with other state-of-the-art methods under different scenarios, including noise-free, additive random noise, and periodic stripe noise. Comparative results indicate that the proposed method has a more well-balanced performance and robustness. The proposed method contributes a valuable reference for solving the difficult problem of noise image matching.

Figures (13)

• Figure 1: The effect of noise on image matching. Query image on the left, reference image on the right. (a) Noise-free feature extraction. (b) Noise-free feature matching. (c) Noise interferes with feature extraction. (d) Noise interferes with incorrect matching.
• Figure 2: The framework of the proposed robust method for solving noise image matching
• Figure 3: Feature pyramid network. As the convolutional layers increase, the resolution of the feature map decreases, whereas the richer the feature semantic information of the characterization will be.
• Figure 4: Feature enhancement transformer. (a) Alternating combinations of self-attention and cross-attention to form the feature enhancement transformer module. (b) Illustration of the multi-head linear attention layer.
• Figure 5: Fine-level coordinate regression using heatmap regression ref65, where ${\left \langle \ {\cdot,\cdot} \ \right \rangle}_F$ denotes the Frobenius inner product; X and Y are normalized grid matrix with $n \times m$, and ${{X}_{i,j}}=\frac{2j-n}{n}, {{Y}_{i,j}}=\frac{2i-m}{m}$, $i=1,2,\ldots ,n$, $j=1,2,\ldots ,m$, $n=m=5$ .