HypRAG: Hyperbolic Dense Retrieval for Retrieval Augmented Generation

TL;DR

Dense retrieval for retrieval-augmented generation often relies on Euclidean embeddings that struggle to preserve language hierarchies, increasing hallucination risk. The authors introduce hyperbolic dense retrieval with two variants, HyTE-FH and HyTE-H, in the Lorentz model ($K<0$), plus a geometry-aware Outward Einstein Midpoint pooling to preserve hierarchical depth during aggregation. Empirical results on MTEB and RAGBench show up to 29% gains in context and answer relevance and a clear radial-depth organization (≈20.2% radii increase from Level 1 to Level 5) that Euclidean baselines do not exhibit, highlighting the impact of geometry as a design choice. The approach achieves these improvements with smaller, more efficient models and provides a principled pooling mechanism, suggesting that hyperbolic inductive bias is crucial for faithful RAG systems and scalable retrieval architectures.

Abstract

Embedding geometry plays a fundamental role in retrieval quality, yet dense retrievers for retrieval-augmented generation (RAG) remain largely confined to Euclidean space. However, natural language exhibits hierarchical structure from broad topics to specific entities that Euclidean embeddings fail to preserve, causing semantically distant documents to appear spuriously similar and increasing hallucination risk. To address these limitations, we introduce hyperbolic dense retrieval, developing two model variants in the Lorentz model of hyperbolic space: HyTE-FH, a fully hyperbolic transformer, and HyTE-H, a hybrid architecture projecting pre-trained Euclidean embeddings into hyperbolic space. To prevent representational collapse during sequence aggregation, we introduce the Outward Einstein Midpoint, a geometry-aware pooling operator that provably preserves hierarchical structure. On MTEB, HyTE-FH outperforms equivalent Euclidean baselines, while on RAGBench, HyTE-H achieves up to 29% gains over Euclidean baselines in context relevance and answer relevance using substantially smaller models than current state-of-the-art retrievers. Our analysis also reveals that hyperbolic representations encode document specificity through norm-based separation, with over 20% radial increase from general to specific concepts, a property absent in Euclidean embeddings, underscoring the critical role of geometric inductive bias in faithful RAG systems.

Figures (4)

• Figure 1: Hierarchies in Text. (A) Documents naturally organize into branching hierarchies where general topics spawn increasingly specific subtopics. Euclidean spaces distort such hierarchies due to crowding effects, while hyperbolic geometry preserves hierarchical relationships through exponential volume growth. (B) Ricci curvature analysis of document embeddings from strong baselines reveals predominantly negative curvature, indicating tree-like semantic structure.
• Figure 2: HyTE Architecture. A) HyTE-FH Encoder Block, B) HyTE-FH architecture, C) HyTE-H Architecture.
• Figure 3: Outward Einstein Midpoint. Size of token shows its contribution towards aggregation.
• Figure 4: Empirical validation of hierarchical encoding.Left: Euclidean models show flat or decreasing norms. Middle: HyTE-H demonstrate increasing norms with fine-tuning enhancing this trend. Right: HyTE-FH achieves +20.2% total increase from L1 to L5. Bottom: Normalized comparison and percent change summary highlighting the contrasting behaviors of different geometric approaches.

Theorems & Definitions (17)

• Definition 4.1: Lorentz Projection
• Definition 4.2: Radial Depth
• Proposition 4.3: Euclidean Mean Contracts
• Definition 4.4: Outward Bias
• Proposition 4.5: Implicit Radial Weighting is Insufficient
• Theorem 4.6: OEM Pre-Projection Bound
• Theorem 4.7: OEM Outward Bias
• Lemma 1.1: Lorentzian Inner Product Bound
• proof
• Proposition 1.2: Euclidean Mean Contracts