Certifying Knowledge Comprehension in LLMs
Isha Chaudhary, Vedaant V. Jain, Gagandeep Singh
TL;DR
The paper tackles the reliability and generalization limits of LLMs on knowledge-comprehension tasks by replacing static benchmarks with specification-driven probability guarantees. It introduces LLMCert-C, a probabilistic certification framework that builds knowledge-graph-based specifications (over PrimeKG and Wikidata5m) incorporating natural noise and multi-hop reasoning, and then estimates the probability $p$ of correct answers using $n$ iid samples to produce Clopper-Pearson intervals with confidence $1-\\delta$. The framework operates in a black-box setting, enabling certification of API-based or closed-source models, and delivers rigorous bounds $[p_l, p_u]$ to compare SOTA models while accounting for prompt noise. Experimental results show vulnerabilities to distractors, establish performance hierarchies across models, and reveal potential degradation due to quantization, underscoring the need for principled, guarantee-driven evaluation in safety-critical deployments. The work provides open-source tooling and demonstrates that specification-based certification can supplant unreliable benchmarking in high-stakes QA tasks.
Abstract
Large Language Models (LLMs) are increasingly deployed in safety-critical systems where they provide answers based on in-context information derived from knowledge bases. As LLMs are increasingly envisioned as superhuman agents, their proficiency in knowledge comprehension-extracting relevant information and reasoning over it to answer questions, a key facet of human intelligence-becomes crucial. However, existing evaluations of LLMs on knowledge comprehension are typically conducted on small test sets, but these datasets represent only a tiny fraction of the vast number of possible queries. Simple empirical evaluations on these limited test sets raises concerns about the reliability and generalizability of the results. In this work, we introduce the first specification and certification framework for knowledge comprehension in LLMs, providing formal probabilistic guarantees for reliability. Instead of a fixed dataset, we design novel specifications that mathematically represent prohibitively large probability distributions of knowledge comprehension prompts with natural noise, using knowledge graphs. From these specifications, we generate quantitative certificates that offer high-confidence, tight bounds on the probability that a given LLM correctly answers any question drawn from the specification distribution. We apply our framework to certify SOTA LLMs in two domains: precision medicine and general question-answering. Our results reveal previously unrecognized vulnerabilities in SOTA LLMs due to natural noise in the prompts. Additionally, we establish performance hierarchies with formal guarantees among the SOTA LLMs, particularly in the context of precision medicine question-answering.