Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Affine steerers for structured keypoint description

Georg Bökman, Johan Edstedt, Michael Felsberg, Fredrik Kahl

TL;DR

A way to train deep learning based keypoint descriptors that makes them approximately equivariant for locally affine transformations of the image plane to generalize the recently introduced concept of steerers from rotations to affine transformations.

Abstract

We propose a way to train deep learning based keypoint descriptors that makes them approximately equivariant for locally affine transformations of the image plane. The main idea is to use the representation theory of GL(2) to generalize the recently introduced concept of steerers from rotations to affine transformations. Affine steerers give high control over how keypoint descriptions transform under image transformations. We demonstrate the potential of using this control for image matching. Finally, we propose a way to finetune keypoint descriptors with a set of steerers on upright images and obtain state-of-the-art results on several standard benchmarks. Code will be published at github.com/georg-bn/affine-steerers.

Affine steerers for structured keypoint description

TL;DR

A way to train deep learning based keypoint descriptors that makes them approximately equivariant for locally affine transformations of the image plane to generalize the recently introduced concept of steerers from rotations to affine transformations.

Abstract

We propose a way to train deep learning based keypoint descriptors that makes them approximately equivariant for locally affine transformations of the image plane. The main idea is to use the representation theory of GL(2) to generalize the recently introduced concept of steerers from rotations to affine transformations. Affine steerers give high control over how keypoint descriptions transform under image transformations. We demonstrate the potential of using this control for image matching. Finally, we propose a way to finetune keypoint descriptors with a set of steerers on upright images and obtain state-of-the-art results on several standard benchmarks. Code will be published at github.com/georg-bn/affine-steerers.
Paper Structure (8 sections, 8 equations, 2 figures)

This paper contains 8 sections, 8 equations, 2 figures.

Table of Contents

  1. Introduction
  2. Related work
  3. Image matching.
  4. Affine correspondences.
  5. Equivariant representation learning.
  6. Background
  7. Method
  8. Representation theory of GL(2)

Figures (2)

  • Figure 1: Qualitative example. We show qualitative matching results of our AffSteer-B descriptor which achieves SotA results for detector-descriptors on several benchmarks for image matching. We plot inliers after homography estimation with RANSAC.
  • Figure 2: Overview of the steering idea for keypoint matching. An affine steerer gives a way to modify descriptions as if they were obtained from warped images, without having to rerun the descriptor network on warped images. The steerer is a linear map and hence computationally light to use.

Theorems & Definitions (2)

  • definition thmcounterdefinition: Group representation
  • definition thmcounterdefinition: Steerer