HyperSpaceX: Radial and Angular Exploration of HyperSpherical Dimensions

TL;DR

HyperSpaceX extends hyperspherical representation learning by exploiting both angular and radial dimensions in multi-hyperspherical spaces. The core is DistArc loss, which combines an angular margin $\cos(\theta_{y_i}+m)$ with radial alignment through $\cos(\phi_{y_i})$, guided by radial proxies $\omega_{r_{y_i}}$ and resultant vectors $R_{y_i} = -\omega_{r_{y_i}} + x_i$, and weighted by $\lambda$ and $\delta$ terms. A predictive measure based on the smallest $||R_i||_2$ is proposed to assess multi-radius proximity, complemented by latent-space visualizations. Empirical results on seven object datasets and six face datasets show SoTA performance, especially at low embedding dimensionality, validating the discriminative benefits of radial-angular feature arrangement.

Abstract

Traditional deep learning models rely on methods such as softmax cross-entropy and ArcFace loss for tasks like classification and face recognition. These methods mainly explore angular features in a hyperspherical space, often resulting in entangled inter-class features due to dense angular data across many classes. In this paper, a new field of feature exploration is proposed known as HyperSpaceX which enhances class discrimination by exploring both angular and radial dimensions in multi-hyperspherical spaces, facilitated by a novel DistArc loss. The proposed DistArc loss encompasses three feature arrangement components: two angular and one radial, enforcing intra-class binding and inter-class separation in multi-radial arrangement, improving feature discriminability. Evaluation of HyperSpaceX framework for the novel representation utilizes a proposed predictive measure that accounts for both angular and radial elements, providing a more comprehensive assessment of model accuracy beyond standard metrics. Experiments across seven object classification and six face recognition datasets demonstrate state-of-the-art (SoTA) results obtained from HyperSpaceX, achieving up to a 20% performance improvement on large-scale object datasets in lower dimensions and up to 6% gain in higher dimensions.

Figures (5)

• Figure 1: Visual concept of the proposed HyperSpaceX framework: It utilizes a novel radial-angular latent space on hyperspherical manifolds to differentiate features effectively. Initially, class features are indistinguishable due to overlap. The proposed DistArc loss exhibits two feature arrangement learning: high inter-class variation employing multi-radial angular arrangement and minimal intra-class distance, leading to highly separable-discriminable class features.
• Figure 2: The geometric interpretation of angular and radial formations in multi-hyperspherical dimensions in the training phase through (a) $\theta$ and angular-margin penalty m, and (b) angle $\phi$ between scaled proxy vector $\omega_{y_{i}}$ and resultant vector $R$. (c) Shows the vector representation of $R$ and $\omega_{y_{i}}$ in a reverse direction for computing angle $\phi$ using cosine of an angle $\phi$.
• Figure 3: The 2-D (in first row) and 3-D (in second row) latent space visualization of features learnt through (a) metric-based loss functions, (b) Angular-softmax-based loss functions, and (c) the proposed Radial-Angular DistArc loss. Further showing the decision-making process of assigning a test sample $x$ to the most favourable class distribution represented using either class center $c$ or proxy vector $\omega$.
• Figure 4: (a) Illustrating comparative visual analysis of the organization of MNIST class feature distribution in the latent space. The feature representations are learned using (i) Cross-entropy loss, (ii) ArcFace loss, and (iii) the proposed DistArc loss on the MNIST database, where each color represents a unique class. While (b) depicts the subclass organization of the CIFAR-100 dataset on 2D multi-hyperspherical manifolds. The color of each line denotes a distinct superclass, facilitating angular separability. Subclasses within each superclass are distinguished radially, with each subclass represented as blobs extending radially from the superclass center.
• Figure 5: Analysis of loss functions using small-class simple datasets (a) MNIST and (b) FashionMNIST, and complex dataset (c) CIFAR-10. The first row visualizes the features, showcasing the outcomes of learning with DistArc loss over 2D multi-spherical manifolds. The last two rows illustrate classification performance of different backbones with a 2-D and 512-D embedding sizes, trained using Cross-entropy and DistArc loss.