Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

SC-Net: Robust Correspondence Learning via Spatial and Cross-Channel Context

Shuyuan Lin, Hailiang Liao, Qiang Qi, Junjie Huang, Taotao Lai, Jian Weng

TL;DR

SC-Net tackles robust two-view correspondence by learning motion fields with bilateral spatial and cross-channel context. It introduces Adaptive Focus Regularization to sharpen local message passing, Bilateral Field Adjustment to model long-range spatial-channel interactions, and a Position-aware Recovery step to recover final motion vectors, collectively reducing over-smoothing and improving discontinuity preservation. The approach yields state-of-the-art results in relative pose estimation and outlier removal on YFCC100M and SUN3D, while demonstrating strong generalization across detectors and datasets with efficient computation. These contributions advance robust correspondence learning for downstream tasks like SLAM and structure-from-motion, offering practical gains in accuracy and robustness across varied scenes.

Abstract

Recent research has focused on using convolutional neural networks (CNNs) as the backbones in two-view correspondence learning, demonstrating significant superiority over methods based on multilayer perceptrons. However, CNN backbones that are not tailored to specific tasks may fail to effectively aggregate global context and oversmooth dense motion fields in scenes with large disparity. To address these problems, we propose a novel network named SC-Net, which effectively integrates bilateral context from both spatial and channel perspectives. Specifically, we design an adaptive focused regularization module (AFR) to enhance the model's position-awareness and robustness against spurious motion samples, thereby facilitating the generation of a more accurate motion field. We then propose a bilateral field adjustment module (BFA) to refine the motion field by simultaneously modeling long-range relationships and facilitating interaction across spatial and channel dimensions. Finally, we recover the motion vectors from the refined field using a position-aware recovery module (PAR) that ensures consistency and precision. Extensive experiments demonstrate that SC-Net outperforms state-of-the-art methods in relative pose estimation and outlier removal tasks on YFCC100M and SUN3D datasets. Source code is available at http://www.linshuyuan.com.

SC-Net: Robust Correspondence Learning via Spatial and Cross-Channel Context

TL;DR

SC-Net tackles robust two-view correspondence by learning motion fields with bilateral spatial and cross-channel context. It introduces Adaptive Focus Regularization to sharpen local message passing, Bilateral Field Adjustment to model long-range spatial-channel interactions, and a Position-aware Recovery step to recover final motion vectors, collectively reducing over-smoothing and improving discontinuity preservation. The approach yields state-of-the-art results in relative pose estimation and outlier removal on YFCC100M and SUN3D, while demonstrating strong generalization across detectors and datasets with efficient computation. These contributions advance robust correspondence learning for downstream tasks like SLAM and structure-from-motion, offering practical gains in accuracy and robustness across varied scenes.

Abstract

Recent research has focused on using convolutional neural networks (CNNs) as the backbones in two-view correspondence learning, demonstrating significant superiority over methods based on multilayer perceptrons. However, CNN backbones that are not tailored to specific tasks may fail to effectively aggregate global context and oversmooth dense motion fields in scenes with large disparity. To address these problems, we propose a novel network named SC-Net, which effectively integrates bilateral context from both spatial and channel perspectives. Specifically, we design an adaptive focused regularization module (AFR) to enhance the model's position-awareness and robustness against spurious motion samples, thereby facilitating the generation of a more accurate motion field. We then propose a bilateral field adjustment module (BFA) to refine the motion field by simultaneously modeling long-range relationships and facilitating interaction across spatial and channel dimensions. Finally, we recover the motion vectors from the refined field using a position-aware recovery module (PAR) that ensures consistency and precision. Extensive experiments demonstrate that SC-Net outperforms state-of-the-art methods in relative pose estimation and outlier removal tasks on YFCC100M and SUN3D datasets. Source code is available at http://www.linshuyuan.com.
Paper Structure (20 sections, 15 equations, 5 figures, 7 tables)

This paper contains 20 sections, 15 equations, 5 figures, 7 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Traditional Outlier Removal Methods
  4. Learning-based Outlier Removal Methods
  5. Methodology
  6. Overview
  7. Adaptive Focused Regularization
  8. Bilateral Field Adjustment
  9. Loss Function
  10. Experiments
  11. Implement Details
  12. Datasets and Evaluation Protocols
  13. Relative Pose Estimation
  14. Outlier Removal
  15. Discussions
  16. ...and 5 more sections

Figures (5)

  • Figure 1: Architecture of the proposed SC-Net. It consists of $L$ rectifying layers, each including AFR, BFA, and PAR. SC-Net takes putative correspondences $P\in\mathbb{R}^{N\times 4}$ as input and predicts the inlier probabilities $\hat{z}_{cls}^{l+1}$ and regression weights $\hat{z}_{reg}^{l+1}$. Purple lines denote propagation to subsequent layer.
  • Figure 2: Structure of Bilateral Field Adjustment (BFA).
  • Figure 3: Structure of Motion Feature Modulator (MFM).
  • Figure 4: Comparison of F-score results for different models under the different logit thresholds on the known (a) and unknown (b) scenes in YFCC100M.
  • Figure 5: Qualitative results of outlier removal (1st and 2nd rows show the outdoor scenes from YFCC100M, while 3rd and 4th rows show the indoor scenes from SUN3D). False matches are marked in red and correct matches in green.