Scale-Free Image Keypoints Using Differentiable Persistent Homology

TL;DR

This paper addresses scale-free keypoint detection by proposing MorseDet, a differentiable detector built on Morse theory and persistent homology. It introduces a novel topological detector loss that uses birth/death times and boundary similarity to align topology-preserving features across images in an unsupervised fashion. MorseDet achieves competitive keypoint repeatability on HPatches, demonstrating robustness to scale and viewpoint changes while avoiding fixed patch densities. The work provides a principled foundation for integrating topology into computer vision and points to future extensions of topology-based losses in data-heavy CV tasks.

Abstract

In computer vision, keypoint detection is a fundamental task, with applications spanning from robotics to image retrieval; however, existing learning-based methods suffer from scale dependency and lack flexibility. This paper introduces a novel approach that leverages Morse theory and persistent homology, powerful tools rooted in algebraic topology. We propose a novel loss function based on the recent introduction of a notion of subgradient in persistent homology, paving the way toward topological learning. Our detector, MorseDet, is the first topology-based learning model for feature detection, which achieves competitive performance in keypoint repeatability and introduces a principled and theoretically robust approach to the problem.

Figures (22)

• Figure 1: The evolution of the sub-level sets of a surface filtered by height, i.e. value on the $z$ axis. As the height crosses $z_1$, a new loop is born in correspondence with a saddle (green point), then the loop changes smoothly until $z$ hits $z_2$, the value of a corresponding maximum (blue point), and the loop disappears. $z_1$ and $z_2$ are, respectively, the topological feature's birth time and death time.
• Figure 2: Pipeline overview: a convolutional neural network (CNN) is employed to generate height maps from input images. During inference, keypoints are efficiently detected as local maxima of these maps, utilizing non-maximum suppression. For training, our detector loss computation involves the application of discrete Morse theory algorithms to compute persistence pairs, and then the maps at the corresponding positions are compared through the boundary similarity. The resulting gradients are subsequently backpropagated.
• Figure : (a)
• Figure : (a)
• Figure : (a)