Honest AI: Fine-Tuning "Small" Language Models to Say "I Don't Know", and Reducing Hallucination in RAG

TL;DR

This work tackles hallucination in language models by evaluating retrieval-augmented generation (RAG) and fine-tuning strategies on the CRAG benchmark, with a focus on resource-efficient, sub-10B models. The authors propose Honest AI, a hybrid approach that fine-tunes small LLMs to answer 'I don't know' when uncertain and combines RAG with domain-aware routing and content pruning to reduce hallucinations. Key findings show that RAG alone yields limited gains on CRAG, while fine-tuning significantly improves accuracy and a hybrid RAG+fine-tuning approach delivers the best overall performance, including top results on false-premise questions. The work demonstrates that small models, when paired with smart retrieval and careful calibration of responses, can achieve robust performance with lower compute, enabling practical enterprise deployment. The results highlight the importance of content quality in retrieval, domain-specific strategies, and adaptive correction for hallucination reduction.

Abstract

Hallucination is a key roadblock for applications of Large Language Models (LLMs), particularly for enterprise applications that are sensitive to information accuracy. To address this issue, two general approaches have been explored: Retrieval-Augmented Generation (RAG) to supply LLMs with updated information as context, and fine-tuning the LLMs with new information and desired output styles. In this paper, we propose Honest AI: a novel strategy to fine-tune "small" language models to say "I don't know" to reduce hallucination, along with several alternative RAG approaches. The solution ranked 1st in Task 2 for the false premise question. The alternative approaches include using RAG with search engine and knowledge graph results, fine-tuning base LLMs with new information and combinations of both approaches. Although all approaches improve the performance of the LLMs, RAG alone does not significantly improve the performance and fine-tuning is needed for better results. Finally, the hybrid approach achieved the highest score in the CRAG benchmark. In addition, our approach emphasizes the use of relatively small models with fewer than 10 billion parameters, promoting resource efficiency.

Figures (5)

• Figure 1: Team Future in Meta CRAG Challenge 2024 Winner List
• Figure 2: The Distribution of Questions in Different Domains and Types of Timeliness
• Figure 3: Fine-tuning Process
• Figure 4: Hybrid Approach
• Figure 5: Team Future Got 64.6% Score in false_premise Question Type