Minimizing Hyperbolic Embedding Distortion with LLM-Guided Hierarchy Restructuring

TL;DR

The paper tackles the sensitivity of hyperbolic embeddings to input hierarchy structure by proposing an LLM-guided, prompt-based restructuring pipeline. It converts hierarchies into compact textual representations, prompts LLMs to favor width and single inheritance while de-emphasizing balance, validates outputs, and then applies contemporary constructive hyperbolic embedding methods. Across 16 diverse hierarchies, this approach yields consistent improvements in average and worst-case distortion, with explainable justifications that help knowledge engineers reason about changes. The work demonstrates practical potential for boosting hyperbolic learning performance and integrating explainability into ontology design and knowledge engineering workflows, while outlining directions for broader LLM applicability and ontology creation.

Abstract

Hyperbolic geometry is an effective geometry for embedding hierarchical data structures. Hyperbolic learning has therefore become increasingly prominent in machine learning applications where data is hierarchically organized or governed by hierarchical semantics, ranging from recommendation systems to computer vision. The quality of hyperbolic embeddings is tightly coupled to the structure of the input hierarchy, which is often derived from knowledge graphs or ontologies. Recent work has uncovered that for an optimal hyperbolic embedding, a high branching factor and single inheritance are key, while embedding algorithms are robust to imbalance and hierarchy size. To assist knowledge engineers in reorganizing hierarchical knowledge, this paper investigates whether Large Language Models (LLMs) have the ability to automatically restructure hierarchies to meet these criteria. We propose a prompt-based approach to transform existing hierarchies using LLMs, guided by known desiderata for hyperbolic embeddings. Experiments on 16 diverse hierarchies show that LLM-restructured hierarchies consistently yield higher-quality hyperbolic embeddings across several standard embedding quality metrics. Moreover, we show how LLM-guided hierarchy restructuring enables explainable reorganizations, providing justifications to knowledge engineers.

Figures (3)

• Figure 1: Prompt-based LLM-Guided hierarchy restructuring approach
• Figure 2: The prompt used for restructuring the hierarchy.
• Figure 3: Hierarchies before (left) and after (right) the LLM-guided restructuring