SLPL SHROOM at SemEval2024 Task 06: A comprehensive study on models ability to detect hallucination

TL;DR

The paper tackles semantic hallucinations in natural language generation, focusing on three SHROOM tasks (machine translation, definition modeling, and paraphrase generation) from SemEval-2024 Task 6. It evaluates three detection paradigms: semantic similarity via LaBSE embeddings and LLM prompts, Natural Language Inference-based entailment scoring using a DeBERTa-v3 model, and an ensemble LLM Judgment Method that leverages multiple agents for meta-reasoning. Results indicate that NLI-based detection provides the strongest signal, while semantic similarity offers moderate performance and the ensemble approach yields mixed outcomes, not surpassing the best baselines. The work highlights persistent challenges in reliably detecting semantic hallucinations and identifies directions for developing more robust, model-agnostic detection strategies.

Abstract

Language models, particularly generative models, are susceptible to hallucinations, generating outputs that contradict factual knowledge or the source text. This study explores methods for detecting hallucinations in three SemEval-2024 Task 6 tasks: Machine Translation, Definition Modeling, and Paraphrase Generation. We evaluate two methods: semantic similarity between the generated text and factual references, and an ensemble of language models that judge each other's outputs. Our results show that semantic similarity achieves moderate accuracy and correlation scores in trial data, while the ensemble method offers insights into the complexities of hallucination detection but falls short of expectations. This work highlights the challenges of hallucination detection and underscores the need for further research in this critical area.

Figures (1)

• Figure 1: Comparison of accuracy and correlation scores across multiple models in model-aware and model-agnostic datasets.