Coarse-to-Fine Highlighting: Reducing Knowledge Hallucination in Large Language Models

TL;DR

COFT is proposed, a novelarse-to-Fine highligh\textbf{F}ine highligh\textbf{T}ing method to focus on different granularity-level key texts, thereby avoiding getting lost in lengthy contexts, leading to a superior performance over $30\% in the F1 score metric.

Abstract

Generation of plausible but incorrect factual information, often termed hallucination, has attracted significant research interest. Retrieval-augmented language model (RALM) -- which enhances models with up-to-date knowledge -- emerges as a promising method to reduce hallucination. However, existing RALMs may instead exacerbate hallucination when retrieving lengthy contexts. To address this challenge, we propose COFT, a novel \textbf{CO}arse-to-\textbf{F}ine highligh\textbf{T}ing method to focus on different granularity-level key texts, thereby avoiding getting lost in lengthy contexts. Specifically, COFT consists of three components: \textit{recaller}, \textit{scorer}, and \textit{selector}. First, \textit{recaller} applies a knowledge graph to extract potential key entities in a given context. Second, \textit{scorer} measures the importance of each entity by calculating its contextual weight. Finally, \textit{selector} selects high contextual weight entities with a dynamic threshold algorithm and highlights the corresponding paragraphs, sentences, or words in a coarse-to-fine manner. Extensive experiments on the knowledge hallucination benchmark demonstrate the effectiveness of COFT, leading to a superior performance over $30\%$ in the F1 score metric. Moreover, COFT also exhibits remarkable versatility across various long-form tasks, such as reading comprehension and question answering.

Figures (12)

• Figure 1: COFT achieves state-of-the-art performance on a broad range of long-form tasks compared with existing methods, using ChatGPT as the backbone.
• Figure 2: An overview of COFT. COFT integrates recaller, scorer, and selector into a unified framework to reduce knowledge hallucination. The workflow is as follows. (1) Perform Named Entity Recognition on the query to extract potential candidate entities. (2) Search the neighboring entities for each potential entity in the knowledge graph to enrich the candidates. (3) Retain candidates that are also present in the reference context as the final key entities. (4) Calculate the contextual weight for each key entity. (5) Calculate the threshold to filter a dynamic proportion of entities. (6) Choose the granularity for highlighting, such as word, sentence, or paragraph. (7) Highlight the reference context based on filtered entities and selected granularity.
• Figure 3: Evaluation on F1 score metric of noise robustness in question answering task, utilizing ChatGPT as the backbone model. COFT demonstrates superior performance on all three open-domain QA benchmarks, especially at higher noise ratios.
• Figure 4: Visualization of the information flow in Vicuna-33B before (left) and after (right) highlighting key lexical units (between two ** symbols). The line color depth reflects the significance of the information flow from the right word to the left.
• Figure 5: Evaluation on EM metric of noise robustness in question answering task, utilizing ChatGPT as the backbone model: COFT demonstrates superior performance on all three open-domain QA benchmarks, especially at higher noise ratios.