DioR: Adaptive Cognitive Detection and Contextual Retrieval Optimization for Dynamic Retrieval-Augmented Generation

TL;DR

DioR addresses persistent hallucinations in retrieval-augmented generation by introducing adaptive cognitive detection to pre-emptively decide when to retrieve and contextual retrieval optimization to refine what and how retrieved information is used. The framework combines Early Detection and Real-time Detection with pre- and post-retrieval steps to guide multi-turn, stepwise retrieval and document segmentation, grounding the LLM's outputs in relevant external knowledge. Across four knowledge-intensive benchmarks and multiple backbones, DioR delivers consistent improvements in exact match, F1, and grounding while reducing hallucinations and improving efficiency. This approach advances practical deployment of dynamic RAG by aligning retrieval triggers with model state and by optimizing the quality and relevance of retrieved content.

Abstract

Dynamic Retrieval-augmented Generation (RAG) has shown great success in mitigating hallucinations in large language models (LLMs) during generation. However, existing dynamic RAG methods face significant limitations in two key aspects: 1) Lack of an effective mechanism to control retrieval triggers, and 2) Lack of effective scrutiny of retrieval content. To address these limitations, we propose an innovative dynamic RAG method, DioR (Adaptive Cognitive Detection and Contextual Retrieval Optimization), which consists of two main components: adaptive cognitive detection and contextual retrieval optimization, specifically designed to determine when retrieval is needed and what to retrieve for LLMs is useful. Experimental results demonstrate that DioR achieves superior performance on all tasks, demonstrating the effectiveness of our work.

Figures (9)

• Figure 1: Dynamic RAG and its limitations about "Timing Selection" and "Document Quality", as well as our solution to address the "Retrieval" limitation.
• Figure 2: Detailed technical framework of DioR. It is mainly divided into two technical aspects: 1) Adaptive cognitive detection; and 2) Contextual retrieval optimization.
• Figure 3: The construction process of Real-time Detection (a) and Early Detection (b) datasets
• Figure 4: An example of an IG attribution score for hallucination and non-hallucination.
• Figure 5: The difference in the distribution of the output entropy values when the model generates the correct answer (non-hallucination) and the wrong answer (hallucination).