From Confidence to Collapse in LLM Factual Robustness

TL;DR

This work targets the robustness of factual knowledge in LLMs by moving beyond accuracy at a fixed decoding condition. It introduces the Factual Robustness Score (FRS), which combines intrinsic uncertainty via entropy with decoding perturbations measured by a breaking temperature $t_b$, yielding a single metric that reflects both initial confidence and resilience to temperature shifts. Across five LLMs and three closed-book QA datasets, FRS reveals that robustness is not solely a function of size; numerical facts tend to be more stable, while some knowledge types remain fragile under perturbation. The results underscore the need for robustness-aware evaluation and offer a foundation for targeted training or retrieval strategies to improve factual retention in real-world deployments.

Abstract

Ensuring the robustness of factual knowledge in LLMs is critical for reliable applications in tasks such as question answering and reasoning. However, existing evaluation methods predominantly focus on performance-based metrics, often investigating from the perspective of prompt perturbations, which captures only the externally triggered side of knowledge robustness. To bridge this gap, we introduce a principled approach to measure factual robustness from the perspective of the generation process by analyzing token distribution entropy in combination with temperature scaling sensitivity. These two factors build the Factual Robustness Score (FRS), a novel metric which quantifies the stability of a fact against perturbations in decoding conditions, given its initial uncertainty. To validate our approach, we conduct extensive experiments on 5 LLMs across 3 closed-book QA datasets (SQuAD, TriviaQA, and HotpotQA). We show that factual robustness varies significantly -- smaller models report an FRS of $0.76$, larger ones $0.93$ -- with accuracy degrading by ~$60\%$ under increased uncertainty. These insights demonstrate how entropy and temperature scaling impact factual accuracy, and lay a foundation for developing more robust knowledge retention and retrieval in future models.

Figures (13)

• Figure 1: Accuracy steadily degrades with increasing temperature levels. We show that temperature is a direct factor in the difficulty of keeping the correct answer. Across all LLMs and datasets, accuracy decreases with increasing temperature, making temperature a direct factor in correct responses.
• Figure 2: Impact of temperature $\mathbf{t}$ on token probability distribution in TriviaQA. As $t$ increases, the probability distribution flattens, reducing certainty in token selection and increasing the likelihood of generating lower-probability responses. This highlights how temperature directly influences response confidence and factual stability.
• Figure 3: Average entropy levels of originally correct answers vs. the breaking temperature levels. While we observe a downward trend, we show that there is only a weak correlation between the initial entropy of an answer and its breaking temperature.
• Figure 4: FRS ($d=1$) over all models, on SQuAD. Although FRS equal to 1 is theoretically achievable when $t_b\to\infty$, in practice, we set it to 1 for all samples where models did not break with $t_b \leq 2$. Hence, the yellow data points.
• Figure 5: Numerical facts are most robust across all models. Grouping the top most robust answers into question categories across datasets, we show the dominance in % of the categories for each model. Answers to numerical or location-based questions are most robust, while questions about names (here: Human) lead to least robust answers.