Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Enhancing Retrieval-Augmented Generation with Topic-Enriched Embeddings: A Hybrid Approach Integrating Traditional NLP Techniques

Rodrigo Kataishi

TL;DR

This work tackles the retrieval precision bottleneck in Retrieval-Augmented Generation (RAG) by introducing topic-enriched embeddings that fuse term-level signals (TF-IDF, LSA) and probabilistic topics (LDA) with contextual embeddings from $\text{all-MiniLM-L6-v2}$ to better ground LLMs in large, thematically diverse corpora. The authors implement a two-phase architecture with parallel lexical and topical streams that are fused into a single embedding via concatenation or weighted averaging, enabling unified indexing and query-time retrieval. Empirical results on a 12,436-document legal corpus show consistent gains in clustering coherence (e.g., Silhouette up to 0.705) and retrieval precision (P@10 up to 0.87, F1@10 up to 0.79) over TF-IDF and purely contextual baselines, validating the approach. The work contributes an open-source framework for embedding enrichment and demonstrates the practical impact of integrating traditional NLP with modern embeddings to improve knowledge-intensive RAG pipelines.

Abstract

Retrieval-augmented generation (RAG) systems rely on accurate document retrieval to ground large language models (LLMs) in external knowledge, yet retrieval quality often degrades in corpora where topics overlap and thematic variation is high. This work proposes topic-enriched embeddings that integrate term-based signals and topic structure with contextual sentence embeddings. The approach combines TF-IDF with topic modeling and dimensionality reduction, using Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA) to encode latent topical organization, and fuses these representations with a compact contextual encoder (all-MiniLM). By jointly capturing term-level and topic-level semantics, topic-enriched embeddings improve semantic clustering, increase retrieval precision, and reduce computational burden relative to purely contextual baselines. Experiments on a legal-text corpus show consistent gains in clustering coherence and retrieval metrics, suggesting that topic-enriched embeddings can serve as a practical component for more reliable knowledge-intensive RAG pipelines.

Enhancing Retrieval-Augmented Generation with Topic-Enriched Embeddings: A Hybrid Approach Integrating Traditional NLP Techniques

TL;DR

This work tackles the retrieval precision bottleneck in Retrieval-Augmented Generation (RAG) by introducing topic-enriched embeddings that fuse term-level signals (TF-IDF, LSA) and probabilistic topics (LDA) with contextual embeddings from to better ground LLMs in large, thematically diverse corpora. The authors implement a two-phase architecture with parallel lexical and topical streams that are fused into a single embedding via concatenation or weighted averaging, enabling unified indexing and query-time retrieval. Empirical results on a 12,436-document legal corpus show consistent gains in clustering coherence (e.g., Silhouette up to 0.705) and retrieval precision (P@10 up to 0.87, F1@10 up to 0.79) over TF-IDF and purely contextual baselines, validating the approach. The work contributes an open-source framework for embedding enrichment and demonstrates the practical impact of integrating traditional NLP with modern embeddings to improve knowledge-intensive RAG pipelines.

Abstract

Retrieval-augmented generation (RAG) systems rely on accurate document retrieval to ground large language models (LLMs) in external knowledge, yet retrieval quality often degrades in corpora where topics overlap and thematic variation is high. This work proposes topic-enriched embeddings that integrate term-based signals and topic structure with contextual sentence embeddings. The approach combines TF-IDF with topic modeling and dimensionality reduction, using Latent Semantic Analysis (LSA) and Latent Dirichlet Allocation (LDA) to encode latent topical organization, and fuses these representations with a compact contextual encoder (all-MiniLM). By jointly capturing term-level and topic-level semantics, topic-enriched embeddings improve semantic clustering, increase retrieval precision, and reduce computational burden relative to purely contextual baselines. Experiments on a legal-text corpus show consistent gains in clustering coherence and retrieval metrics, suggesting that topic-enriched embeddings can serve as a practical component for more reliable knowledge-intensive RAG pipelines.
Paper Structure (17 sections, 7 equations, 3 figures, 3 tables)

This paper contains 17 sections, 7 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Topic Modeling Approaches
  4. Embedding Techniques
  5. Retrieval-Augmented Generation (RAG)
  6. Methodology
  7. Embedding Enrichment: An Innovative Integration of Topic and Contextual Signals
  8. Experiments
  9. Dataset and Preprocessing
  10. Baseline Methods
  11. Evaluation Framework
  12. Evaluation Framework
  13. Results
  14. Retrieval and Clustering Performance
  15. Retrieval Performance
  16. ...and 2 more sections

Figures (3)

  • Figure 1: System architecture of the topic-enriched embeddings for RAG pipeline.
  • Figure 2: t-SNE Visualization of Document Clusters for Different Embedding Techniques
  • Figure 3: Precision–Recall Curves for Different Embedding Techniques.