Optimizing Query Generation for Enhanced Document Retrieval in RAG

TL;DR

This work tackles hallucinations in retrieval-augmented generation by focusing on query quality. It introduces Query Optimization using Query Expansion (QOQA), which uses a top-k query-document alignment score to iteratively refine LLM-generated rephrasings and select highly aligned queries for retrieval. The approach leverages BM25, dense embeddings, and a hybrid score to guide query optimization, and is evaluated on BEIR datasets (SciFact, Trec-Covid, FiQA), showing consistent retrieval improvements and an average accuracy gain of $1.6\%$. The findings underscore the importance of precise, alignment-aware query generation for reducing hallucinations and improving the reliability of RAG-based systems in practical applications.

Abstract

Large Language Models (LLMs) excel in various language tasks but they often generate incorrect information, a phenomenon known as "hallucinations". Retrieval-Augmented Generation (RAG) aims to mitigate this by using document retrieval for accurate responses. However, RAG still faces hallucinations due to vague queries. This study aims to improve RAG by optimizing query generation with a query-document alignment score, refining queries using LLMs for better precision and efficiency of document retrieval. Experiments have shown that our approach improves document retrieval, resulting in an average accuracy gain of 1.6%.

Figures (2)

• Figure 1: Concept figure of QOQA. Given expansion query with top-k docs, we add top-3 rephrased queries and scores to LLM. We optimize the query based on the scores and generate the rephrased query.
• Figure 2: Prompt template used in QOQA. The black texts describe instructions of the optimizing task. The blue texts are original query with top-$N$ retrieved documents with the original query. The purple texts are revised queries by LLM optimizer and scores.