Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuits

TL;DR

This work tackles the unreliability of LLMs in judging their own outputs by introducing Gnosis, a lightweight, fixed-budget self-awareness mechanism that decodes internal traces from frozen LLMs to predict correctness without external supervision. Gnosis combines hidden-state and attention-circuit encoders to produce compact descriptors, enabling near-zero inference cost and only ~5M additional parameters. It achieves state-of-the-art discrimination and calibration across math reasoning, open-domain QA, and academic knowledge benchmarks, with zero-shot transfer across model scales and substantial compute savings compared to external reward models. Additionally, Gnosis enables early detection of failing trajectories, supporting compute-aware control and efficient deployment of reliable language systems.

Abstract

Large language models (LLMs) generate fluent and complex outputs but often fail to recognize their own mistakes and hallucinations. Existing approaches typically rely on external judges, multi-sample consistency, or text-based self-critique, which incur additional compute or correlate weakly with true correctness. We ask: can LLMs predict their own failures by inspecting internal states during inference? We introduce Gnosis, a lightweight self-awareness mechanism that enables frozen LLMs to perform intrinsic self-verification by decoding signals from hidden states and attention patterns. Gnosis passively observes internal traces, compresses them into fixed-budget descriptors, and predicts correctness with negligible inference cost, adding only ~5M parameters and operating independently of sequence length. Across math reasoning, open-domain question answering, and academic knowledge benchmarks, and over frozen backbones ranging from 1.7B to 20B parameters, Gnosis consistently outperforms strong internal baselines and large external judges in both accuracy and calibration. Moreover, it generalizes zero-shot to partial generations, enabling early detection of failing trajectories and compute-aware control. These results show that reliable correctness cues are intrinsic to generation process and can be extracted efficiently without external supervision.

Figures (7)

• Figure 1: Overview of our Gnosis self-awareness mechanism and its performance. Left: Gnosis taps hidden states and attention maps from a frozen LLM, learns to compress them into hidden/attention descriptors, and predicts a scalar correctness (hallucination) score with only $\sim$5 million extra parameters and essentially zero added inference cost. Right: Gnosis surpasses 8B Skywork reward models and a Gemini 2.5 Pro judge (hundreds-of-billions parameters) in AUROC on math reasoning, open-domain QA, and academic knowledge; scores are averaged over the tested backbones listed in Table \ref{['table1']}.
• Figure 2: Early Correctness Prediction on Math-Reasoning. Gnosis (red) achieves higher accuracy and better calibration than both MLP-Prob (blue) and a reward model SkyworkRM-Qwen3-8B (yellow). Notably, after seeing 40% of the completion, Gnosis already matches the full-solution performance of the other methods.
• Figure 3: 2D Embeddings of Features Learned by Gnosis on Math-Reasoning. We show dimensionality-reduced embeddings of hidden-state features (left), attention features (middle), and their merged features (right), with KDE contours and marginal densities for wrong (red) and correct (blue) answers. Hidden features exhibit the clearest separation, attention features show a weaker but still clear separation, and the merged space yields the sharpest overall discrimination between correct and wrong solutions.
• Figure 4: Predicted Correctness Score Distributions. Gnosis (top) displays sharp, bimodal separation between correct (blue) and wrong (red) answers. In contrast, the larger Skywork model (bottom) exhibits diffuse distributions with significant overlap, reflecting higher uncertainty.
• Figure 5: Detailed architecture of the Gnosis encoders. The Hidden Circuit Encoder (left) summarizes projected final-layer hidden states using local temporal mixing followed by global set aggregation. The Attention Circuit Encoder (right) extracts features from downsampled attention maps and models cross-layer/head structure with a lightweight axial processor. Both streams output compact, fixed-size descriptors ($z_{\text{hid}}$ and $z_{\text{attn}}$).