Large Language Models Enhanced Hyperbolic Space Recommender Systems

TL;DR

HyperLLM tackles the challenge of capturing hierarchical user–item structure in recommender systems by using Large Language Models to extract both structural and semantic hierarchies and integrating them into hyperbolic space through a model-agnostic framework. A two-phase training scheme, featuring a meta-optimized semantic extraction via a Mixture of Experts and a structural–semantic integration step, bridges semantic and collaborative spaces within hyperbolic geometry. Empirical results across three Amazon datasets show consistent, substantial gains over strong hyperbolic baselines and other LLM-based methods, with notable improvements in sparse/long-tail settings and faster convergence. The work highlights the importance of hierarchical information in recommendation and offers a robust approach to semantic fusion in hyperbolic space with practical implications for scalable, knowledge-rich recommender systems.

Abstract

Large Language Models (LLMs) have attracted significant attention in recommender systems for their excellent world knowledge capabilities. However, existing methods that rely on Euclidean space struggle to capture the rich hierarchical information inherent in textual and semantic data, which is essential for capturing user preferences. The geometric properties of hyperbolic space offer a promising solution to address this issue. Nevertheless, integrating LLMs-based methods with hyperbolic space to effectively extract and incorporate diverse hierarchical information is non-trivial. To this end, we propose a model-agnostic framework, named HyperLLM, which extracts and integrates hierarchical information from both structural and semantic perspectives. Structurally, HyperLLM uses LLMs to generate multi-level classification tags with hierarchical parent-child relationships for each item. Then, tag-item and user-item interactions are jointly learned and aligned through contrastive learning, thereby providing the model with clear hierarchical information. Semantically, HyperLLM introduces a novel meta-optimized strategy to extract hierarchical information from semantic embeddings and bridge the gap between the semantic and collaborative spaces for seamless integration. Extensive experiments show that HyperLLM significantly outperforms recommender systems based on hyperbolic space and LLMs, achieving performance improvements of over 40%. Furthermore, HyperLLM not only improves recommender performance but also enhances training stability, highlighting the critical role of hierarchical information in recommender systems.

Figures (5)

• Figure 1: The overall architecture of our proposed HyperLLM. It consists of three modules: LLMs-based Structural Extraction, Meta-optimized Semantic Extraction, and Structural and Semantic Integration. These modules operate in the specified order.
• Figure 2: The ablation results for HyperLLM, with missing values for the w/o Meta variant in HICF and HyperCL due to NaN occurrences during the first epoch of training.
• Figure 3: Recall@20 of baselines, HyperLLM without the meta-optimized strategy, and HyperLLM on the validation set at different epochs.
• Figure 4: The Recall@20 results for baselines and baselines with HyperLLM on user groups with different levels of sparsity. G1 to G5 represent user groups 1 to 5, with smaller numbers indicating higher sparsity. The right y-axis represents the percentage improvement of HyperLLM compared to the baseline.
• Figure 5: Hierarchical clustering visualization of representations for HRCF and HRCF with HyperLLM.