Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Adaptive Hyperbolic Kernels: Modulated Embedding in de Branges-Rovnyak Spaces

Leping Si, Meimei Yang, Hui Xue, Shipeng Zhu, Pengfei Fang

TL;DR

This work tackles distortion and rigidity in hyperbolic kernel methods by introducing a curvature-aware de Branges-Rovnyak RKHS that is isometric to the Poincaré ball $\mathbb{D}^n(c)$ and supports an adjustable multiplier to select the appropriate RKHS for any curvature $-c$. It develops a family of adaptive hyperbolic kernels, including the novel adaptive hyperbolic radial kernel (AHRad), designed to modulate hyperbolic features through learnable coefficients $\alpha_l$ and a base cosine-similarity kernel in the de Branges-Rovnyak space. The authors provide rigorous PD guarantees via the multiplier space and demonstrate empirical gains across few-shot, zero-shot, and semantic textual similarity tasks in vision and language. The approach yields lower-distortion embeddings and task-aware representations, offering practical improvements for hierarchical data modeling in cross-domain ML applications.

Abstract

Hierarchical data pervades diverse machine learning applications, including natural language processing, computer vision, and social network analysis. Hyperbolic space, characterized by its negative curvature, has demonstrated strong potential in such tasks due to its capacity to embed hierarchical structures with minimal distortion. Previous evidence indicates that the hyperbolic representation capacity can be further enhanced through kernel methods. However, existing hyperbolic kernels still suffer from mild geometric distortion or lack adaptability. This paper addresses these issues by introducing a curvature-aware de Branges-Rovnyak space, a reproducing kernel Hilbert space (RKHS) that is isometric to a Poincare ball. We design an adjustable multiplier to select the appropriate RKHS corresponding to the hyperbolic space with any curvature adaptively. Building on this foundation, we further construct a family of adaptive hyperbolic kernels, including the novel adaptive hyperbolic radial kernel, whose learnable parameters modulate hyperbolic features in a task-aware manner. Extensive experiments on visual and language benchmarks demonstrate that our proposed kernels outperform existing hyperbolic kernels in modeling hierarchical dependencies.

Adaptive Hyperbolic Kernels: Modulated Embedding in de Branges-Rovnyak Spaces

TL;DR

This work tackles distortion and rigidity in hyperbolic kernel methods by introducing a curvature-aware de Branges-Rovnyak RKHS that is isometric to the Poincaré ball and supports an adjustable multiplier to select the appropriate RKHS for any curvature . It develops a family of adaptive hyperbolic kernels, including the novel adaptive hyperbolic radial kernel (AHRad), designed to modulate hyperbolic features through learnable coefficients and a base cosine-similarity kernel in the de Branges-Rovnyak space. The authors provide rigorous PD guarantees via the multiplier space and demonstrate empirical gains across few-shot, zero-shot, and semantic textual similarity tasks in vision and language. The approach yields lower-distortion embeddings and task-aware representations, offering practical improvements for hierarchical data modeling in cross-domain ML applications.

Abstract

Hierarchical data pervades diverse machine learning applications, including natural language processing, computer vision, and social network analysis. Hyperbolic space, characterized by its negative curvature, has demonstrated strong potential in such tasks due to its capacity to embed hierarchical structures with minimal distortion. Previous evidence indicates that the hyperbolic representation capacity can be further enhanced through kernel methods. However, existing hyperbolic kernels still suffer from mild geometric distortion or lack adaptability. This paper addresses these issues by introducing a curvature-aware de Branges-Rovnyak space, a reproducing kernel Hilbert space (RKHS) that is isometric to a Poincare ball. We design an adjustable multiplier to select the appropriate RKHS corresponding to the hyperbolic space with any curvature adaptively. Building on this foundation, we further construct a family of adaptive hyperbolic kernels, including the novel adaptive hyperbolic radial kernel, whose learnable parameters modulate hyperbolic features in a task-aware manner. Extensive experiments on visual and language benchmarks demonstrate that our proposed kernels outperform existing hyperbolic kernels in modeling hierarchical dependencies.

Paper Structure

This paper contains 46 sections, 7 theorems, 25 equations, 3 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Hyperbolic Kernel Learning
  4. Preliminaries and Notations
  5. Notations
  6. Poincaré Ball Model
  7. Adaptive Hyperbolic Kernels
  8. Curvature-aware De Branges-Rovnyak Space
  9. Contractive Space Structure
  10. Generalization to Arbitrary Curvature
  11. Proposed Hyperbolic Kernels
  12. Design of the Multiplier $\bm{b}$
  13. Design of Hyperbolic Kernels
  14. Experiments
  15. Few-Shot Learning
  16. ...and 31 more sections

Key Result

Proposition 1

sautel2022some A function $\bm{b}$ belongs to a multiplier in $\mathcal{M}(\mathcal{H}^2_n \otimes \mathcal{D}, \mathcal{H}^2_n)$ if and only if: for some $\varepsilon>0$. The infimum of such constants $\varepsilon$ is known as the multiplier norm of $\bm{b}$, denoted by $\Vert\bm{b}\Vert_{\mathcal{M}(\mathcal{H}^2_n \otimes \mathcal{D}, \mathcal{H}^2_n)}$.

Figures (3)

  • Figure 1: Embedding of the same tree characterized by identical branching angles and branch lengths (with hierarchical node structure) in Euclidean and hyperbolic spaces. The left figure shows the embedding in Euclidean space, where some branches overlap. The right figure illustrates the embedding in hyperbolic space, where the exponential expansion property enables distortion-free embedding of the tree.
  • Figure 2: Bar chart visualization of the parameters in Eq. \ref{['eq:AHRad']}, where the x-axis represents the index $l$ of the coefficients, and the y-axis denotes the magnitude of $a_l$
  • Figure 3: Visualization of extracted features in the zero-shot learning setting on the AWA2 unseen dataset, including visual features from ten classes and the corresponding semantic prototype for each class.

Theorems & Definitions (12)

  • Definition 1
  • Proposition 1
  • Proposition 2
  • Lemma 1
  • Theorem 1
  • Definition 2
  • Lemma
  • proof
  • Theorem
  • proof
  • ...and 2 more