Comprehensive Comparison of RAG Methods Across Multi-Domain Conversational QA

TL;DR

The paper addresses the lack of systematic evaluation of retrieval-augmented generation (RAG) for multi-turn conversational QA. It conducts a unified empirical study across eight diverse datasets, comparing vanilla RAG, basic retrieval strategies, and several advanced methods, while assessing both retrieval quality and answer generation. Key findings show that robust, simpler approaches like HyDE, Hybrid BM25, and Reranker frequently outperform vanilla RAG, and that dataset structure and dialogue length critically shape effectiveness. The work emphasizes that practical gains hinge on aligning retrieval strategy with dataset characteristics rather than indiscriminately increasing model complexity, providing actionable guidance and releasing the code for reproducibility.

Abstract

Conversational question answering increasingly relies on retrieval-augmented generation (RAG) to ground large language models (LLMs) in external knowledge. Yet, most existing studies evaluate RAG methods in isolation and primarily focus on single-turn settings. This paper addresses the lack of a systematic comparison of RAG methods for multi-turn conversational QA, where dialogue history, coreference, and shifting user intent substantially complicate retrieval. We present a comprehensive empirical study of vanilla and advanced RAG methods across eight diverse conversational QA datasets spanning multiple domains. Using a unified experimental setup, we evaluate retrieval quality and answer generation using generator and retrieval metrics, and analyze how performance evolves across conversation turns. Our results show that robust yet straightforward methods, such as reranking, hybrid BM25, and HyDE, consistently outperform vanilla RAG. In contrast, several advanced techniques fail to yield gains and can even degrade performance below the No-RAG baseline. We further demonstrate that dataset characteristics and dialogue length strongly influence retrieval effectiveness, explaining why no single RAG strategy dominates across settings. Overall, our findings indicate that effective conversational RAG depends less on method complexity than on alignment between the retrieval strategy and the dataset structure. We publish the code used.\footnote{\href{https://github.com/Klejda-A/exp-rag.git}{GitHub Repository}}

Figures (6)

• Figure 1: Conversational Search Problem. One sample from the INSCIT dataset wu_inscit_2023.
• Figure 2: Theoretical minimum and maximum ranges of F1 that can be achieved with the LLM. Minimum is achieved by No RAG method, which retrieves no contexts, whereas the maximum is achieved by the Oracle Context method, which directly uses the gold label context.
• Figure 3: Relationship between retriever (MRR@5) and generator (F1) performance for each dataset and method.
• Figure 4: MRR performance across conversational turn for each dataset using Vanilla RAG. The annotations indicate the number of samples per turn for each dataset.
• Figure 5: List of the system prompts used for each dataset.