MetaCheckGPT -- A Multi-task Hallucination Detector Using LLM Uncertainty and Meta-models

TL;DR

Hallucinations in LLMs threaten reliability; this work introduces a meta-regressor framework that aggregates uncertainty signals from a diverse ensemble of LLMs to detect hallucinations and guide model evaluation/integration, using $\rho_s$ (Spearman correlation) and $\epsilon$ (MAE) as optimization targets. The approach compares generated sentences against stochastically generated responses without external databases and trains meta-models via meta-search cross-validation, exploring ensemble methods. It achieves top results on SHROOM Task-6 across model-agnostic and model-aware tracks and includes an error analysis contrasting GPT-4 with the best model. The findings suggest that ensemble uncertainty signals yield robust hallucination detection, with practical implications for safer deployment and potential integration with retrieval-augmented knowledge sources.

Abstract

Hallucinations in large language models (LLMs) have recently become a significant problem. A recent effort in this direction is a shared task at Semeval 2024 Task 6, SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakes. This paper describes our winning solution ranked 1st and 2nd in the 2 sub-tasks of model agnostic and model aware tracks respectively. We propose a meta-regressor framework of LLMs for model evaluation and integration that achieves the highest scores on the leaderboard. We also experiment with various transformer-based models and black box methods like ChatGPT, Vectara, and others. In addition, we perform an error analysis comparing GPT4 against our best model which shows the limitations of the former.

Figures (2)

• Figure 1: MetaCheckGPT: Generated sentences are compared against stochastically generated responses.
• Figure 2: Hallucination examples for each task type