Credal Transformer: A Principled Approach for Quantifying and Mitigating Hallucinations in Large Language Models

TL;DR

The paper addresses hallucinations in LLMs by arguing that Softmax-based attention creates artificial certainty. It introduces the Credal Transformer, which replaces standard attention with a Credal Attention Mechanism (CAM) that outputs a credal set over attention distributions via a Dirichlet-based evidential model, with per-head uncertainty measured by vacuity $U_i = L/\alpha_{i0}$. The approach enables out-of-distribution detection, ambiguity quantification, and abstention on unanswerable questions, while maintaining near-identical computational efficiency. This work establishes a principled design paradigm for uncertainty-aware AI that integrates epistemic uncertainty directly into the model’s reasoning process.

Abstract

Large Language Models (LLMs) hallucinate, generating factually incorrect yet confident assertions. We argue this stems from the Transformer's Softmax function, which creates "Artificial Certainty" by collapsing ambiguous attention scores into a single probability distribution, discarding uncertainty information at each layer. To fix this, we introduce the Credal Transformer, which replaces standard attention with a Credal Attention Mechanism (CAM) based on evidential theory. CAM produces a "credal set" (a set of distributions) instead of a single attention vector, with the set's size directly measuring model uncertainty. We implement this by re-conceptualizing attention scores as evidence masses for a Dirichlet distribution: sufficient evidence recovers standard attention, while insufficient evidence yields a diffuse distribution, representing ambiguity. Empirically, the Credal Transformer identifies out-of-distribution inputs, quantifies ambiguity, and significantly reduces confident errors on unanswerable questions by abstaining. Our contribution is a new architecture to mitigate hallucinations and a design paradigm that integrates uncertainty quantification directly into the model, providing a foundation for more reliable AI.

Figures (1)

• Figure 1: Performance comparison between standard Softmax Attention and our Credal Attention Mechanism (CAM). The results show that CAM achieves its uncertainty quantification capabilities with negligible computational overhead. Both inference and training times are nearly identical to the highly optimized standard Transformer.