The magnitude vector of images

TL;DR

This work extends the magnitude invariant from metric spaces to images by defining a magnitude vector that assigns per-pixel contributions and linking magnitude-based boundaries to edges. It provides a rigorous mathematical framework, deriving explicit 1D results and practical 2D approximations, and introduces a patch-based speedup alongside a learnable pullback metric to enable scalable edge detection. Empirical results show magnitude-based edge maps are competitive with Sobel and can exhibit distinct topological structures, suggesting complementary information for vision pipelines. Overall, the paper lays the groundwork for magnitude-informed image analysis with practical computational strategies and a learning component that can be integrated into ML workflows.

Abstract

The magnitude of a finite metric space has recently emerged as a novel invariant quantity, allowing to measure the effective size of a metric space. Despite encouraging first results demonstrating the descriptive abilities of the magnitude, such as being able to detect the boundary of a metric space, the potential use cases of magnitude remain under-explored. In this work, we investigate the properties of the magnitude on images, an important data modality in many machine learning applications. By endowing each individual images with its own metric space, we are able to define the concept of magnitude on images and analyse the individual contribution of each pixel with the magnitude vector. In particular, we theoretically show that the previously known properties of boundary detection translate to edge detection abilities in images. Furthermore, we demonstrate practical use cases of magnitude for machine learning applications and propose a novel magnitude model that consists of a computationally efficient magnitude computation and a learnable metric. By doing so, we address the computational hurdle that used to make magnitude impractical for many applications and open the way for the adoption of magnitude in machine learning research.

Figures (6)

• Figure 1: An illustration of the magnitude calculation of a two-channel, two-pixel, one-dimensional image. The solid lines represent the step functions, the dashed blue line is the numerical magnitude and the dotted orange line represents the theoretical magnitude.
• Figure 2: Two 1D images. The brightness channel is constant across at least one of the dimensions.
• Figure 3: Benchmark experiments performed on the $50$ test imaged of BIPEDv2. We test the computational speedup as well as the the approximation quality of Algorithm \ref{['alg:speedup']} and the rank-1 and local approximations outlined in Subsection \ref{['subsec:2d_images']}.
• Figure 4: A graphical overview of the magnitude edge detector. During training, we train the autoencoder (optionally the feature extractor) as presented in \ref{['subfig:edge_detector']} and during inference, we use the image transformer of \ref{['subfig:image_transformer']}
• Figure 5: Example edge maps taken from the test set of BIPEDv2. We see that the ground truth annotation of the images not always exact and, therefore, any pixel-level evaluation should be taken with care. We compare the Sobel filter output, our best-performing magnitude model and the local approximation of the vanilla magnitude. The colours have been inverted for better visibility.

Theorems & Definitions (30)

• Definition 2.1
• Definition 2.2
• Definition 2.3
• Definition 2.4: willerton2014magnitude
• Definition 2.5: Digital Image
• Definition 2.6: Domain
• Definition 2.7: Analogue Image
• Remark 2.8
• Definition 2.9: Digitised Image
• Remark 2.10: Notation