Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Introducing a new hyper-parameter for RAG: Context Window Utilization

Kush Juvekar, Anupam Purwar

TL;DR

This paper introduces a new hyper-parameter for Retrieval-Augmented Generation systems called Context Window Utilization, and reveals that an optimal chunk size balances the trade-off between providing sufficient context and minimizing irrelevant information.

Abstract

This paper introduces a new hyper-parameter for Retrieval-Augmented Generation (RAG) systems called Context Window Utilization. RAG systems enhance generative models by incorporating relevant information retrieved from external knowledge bases, improving the factual accuracy and contextual relevance of generated responses. The size of the text chunks retrieved and processed is a critical factor influencing RAG performance. This study aims to identify the optimal chunk size that maximizes answer generation quality. Through systematic experimentation, we analyze the effects of varying chunk sizes on the efficiency and effectiveness of RAG frameworks. Our findings reveal that an optimal chunk size balances the trade-off between providing sufficient context and minimizing irrelevant information. These insights are crucial for enhancing the design and implementation of RAG systems, underscoring the importance of selecting an appropriate chunk size to achieve superior performance.

Introducing a new hyper-parameter for RAG: Context Window Utilization

TL;DR

This paper introduces a new hyper-parameter for Retrieval-Augmented Generation systems called Context Window Utilization, and reveals that an optimal chunk size balances the trade-off between providing sufficient context and minimizing irrelevant information.

Abstract

This paper introduces a new hyper-parameter for Retrieval-Augmented Generation (RAG) systems called Context Window Utilization. RAG systems enhance generative models by incorporating relevant information retrieved from external knowledge bases, improving the factual accuracy and contextual relevance of generated responses. The size of the text chunks retrieved and processed is a critical factor influencing RAG performance. This study aims to identify the optimal chunk size that maximizes answer generation quality. Through systematic experimentation, we analyze the effects of varying chunk sizes on the efficiency and effectiveness of RAG frameworks. Our findings reveal that an optimal chunk size balances the trade-off between providing sufficient context and minimizing irrelevant information. These insights are crucial for enhancing the design and implementation of RAG systems, underscoring the importance of selecting an appropriate chunk size to achieve superior performance.
Paper Structure (11 sections, 1 equation, 4 figures, 1 table)

This paper contains 11 sections, 1 equation, 4 figures, 1 table.

Table of Contents

  1. Introduction
  2. Methodology
  3. Dataset Preparation
  4. Why custom datasets?
  5. Evaluation
  6. Results and Discussion
  7. Context Window Utilization
  8. Significance and Conclusion
  9. Limitations and Future Work
  10. Acknowledgement
  11. Custom Dataset

Figures (4)

  • Figure 1: Semantic Similarity Comparisons
  • Figure 2: Number of chunks for which semantic similarity with GPT-4 Turbo answers is maximum in the distinct datasets. The chunk size for each dataset where scores are maximized are also mentioned in brackets.
  • Figure 3: Chunk Size vs. Context Window Utilization across document types with annotated similarity scores for Llama3-70B-Instruct
  • Figure 4: Chunk Size vs. Context Window Utilization across document types with annotated similarity scores for Mistral Large 2