Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Osiris: A Lightweight Open-Source Hallucination Detection System

Alex Shan, John Bauer, Christopher D. Manning

TL;DR

The paper tackles hallucination in retrieval-augmented generation by introducing Osiris-7B, a lightweight open-source detector tailored for multi-hop reasoning. It trains on a perturbed multi-hop QA dataset derived from MuSiQue and musique-v1.0 to enforce explicit evidence chaining, achieving substantial recall gains over GPT-4o on the RAGTruth benchmark while maintaining competitive precision and faster inference on 7B-scale hardware. Key contributions include the data perturbation pipeline, structured reasoning framework, and a comprehensive evaluation showing strong recall improvements (up to ~23%) and real-time inference performance. The work aims to enable scalable, open-source hallucination detection in industry settings, with open-source releases and guidelines for deployment, though it notes limitations in domain coverage and precision trade-offs.

Abstract

Retrieval-Augmented Generation (RAG) systems have gained widespread adoption by application builders because they leverage sources of truth to enable Large Language Models (LLMs) to generate more factually sound responses. However, hallucinations, instances of LLM responses that are unfaithful to the provided context, often prevent these systems from being deployed in production environments. Current hallucination detection methods typically involve human evaluation or the use of closed-source models to review RAG system outputs for hallucinations. Both human evaluators and closed-source models suffer from scaling issues due to their high costs and slow inference speeds. In this work, we introduce a perturbed multi-hop QA dataset with induced hallucinations. Via supervised fine-tuning on our dataset, we achieve better recall with a 7B model than GPT-4o on the RAGTruth hallucination detection benchmark and offer competitive performance on precision and accuracy, all while using a fraction of the parameters. Code is released at our repository.

Osiris: A Lightweight Open-Source Hallucination Detection System

TL;DR

The paper tackles hallucination in retrieval-augmented generation by introducing Osiris-7B, a lightweight open-source detector tailored for multi-hop reasoning. It trains on a perturbed multi-hop QA dataset derived from MuSiQue and musique-v1.0 to enforce explicit evidence chaining, achieving substantial recall gains over GPT-4o on the RAGTruth benchmark while maintaining competitive precision and faster inference on 7B-scale hardware. Key contributions include the data perturbation pipeline, structured reasoning framework, and a comprehensive evaluation showing strong recall improvements (up to ~23%) and real-time inference performance. The work aims to enable scalable, open-source hallucination detection in industry settings, with open-source releases and guidelines for deployment, though it notes limitations in domain coverage and precision trade-offs.

Abstract

Retrieval-Augmented Generation (RAG) systems have gained widespread adoption by application builders because they leverage sources of truth to enable Large Language Models (LLMs) to generate more factually sound responses. However, hallucinations, instances of LLM responses that are unfaithful to the provided context, often prevent these systems from being deployed in production environments. Current hallucination detection methods typically involve human evaluation or the use of closed-source models to review RAG system outputs for hallucinations. Both human evaluators and closed-source models suffer from scaling issues due to their high costs and slow inference speeds. In this work, we introduce a perturbed multi-hop QA dataset with induced hallucinations. Via supervised fine-tuning on our dataset, we achieve better recall with a 7B model than GPT-4o on the RAGTruth hallucination detection benchmark and offer competitive performance on precision and accuracy, all while using a fraction of the parameters. Code is released at our repository.
Paper Structure (24 sections, 2 figures, 4 tables)

This paper contains 24 sections, 2 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Methodology
  3. Dataset
  4. Source
  5. Structured Reasoning for Hallucination Detection
  6. Limitations of Traditional Verification
  7. Shortcut-Driven Reasoning
  8. Hard Distractors and Contrastive Questions
  9. Construction
  10. Example
  11. Evaluation
  12. Training Details
  13. Experiments
  14. RAGTruth
  15. Inference Speed
  16. ...and 9 more sections

Figures (2)

  • Figure 1: Distribution of Token Context Lengths
  • Figure 2: Q/A Dataset Perturbation Pipeline