KG-FPQ: Evaluating Factuality Hallucination in LLMs with Knowledge Graph-based False Premise Questions

TL;DR

KG-FPQ introduces an automated, scalable pipeline to construct FPQs from knowledge graphs and GPTs, addressing factuality hallucination in LLMs. The resulting Knowledge Graph-based False Premise Questions benchmark comprises about $1.78\times 10^5$ FPQs across three domains, six confusability levels, and two task formats, plus FPQ-Judge for automated generative evaluation. Key findings show that FPQ confusability (distance and associations) and task format significantly affect susceptibility, while domain proficiency does not consistently predict resistance; larger models generally fare better, with notable exceptions. The work delivers a practical red-teaming resource and insights for improving factual robustness of LLMs in real-world deployments.

Abstract

Recent studies have demonstrated that large language models (LLMs) are susceptible to being misled by false premise questions (FPQs), leading to errors in factual knowledge, know as factuality hallucination. Existing benchmarks that assess this vulnerability primarily rely on manual construction, resulting in limited scale and lack of scalability. In this work, we introduce an automated, scalable pipeline to create FPQs based on knowledge graphs (KGs). The first step is modifying true triplets extracted from KGs to create false premises. Subsequently, utilizing the state-of-the-art capabilities of GPTs, we generate semantically rich FPQs. Based on the proposed method, we present a comprehensive benchmark, the Knowledge Graph-based False Premise Questions (KG-FPQ), which contains approximately 178k FPQs across three knowledge domains, at six levels of confusability, and in two task formats. Using KG-FPQ, we conduct extensive evaluations on several representative LLMs and provide valuable insights. The KG-FPQ dataset and code are available at~https://github.com/yanxuzhu/KG-FPQ.

Figures (15)

• Figure 1: Top: LLM correctly answers when faced with a TPQ. Middle: LLM experiences factuality hallucination when faced with an FPQ. Bottom: An example of editing triplets in the KG.
• Figure 2: Overview of the construction process of KG-FPQ.
• Figure 3: An illustration of editing methods in KG-FPQ. We use acronyms to refer each method: Neighbor-Same-Concept (NSC), Neighbor-Different-Concept (NDC), Not-Neighbor-Same-Concept (NNSC), Not-Neighbor-Different-Concept (NNDC), Not-Neighbor-Same-Relation (NNSR), Not-Neighbor-Different-Relation (NNDR).
• Figure 4: Overview of the evaluation procedure.
• Figure 5: The average accuracy of all models comparison between NSC and NNSC.