The Earth is Flat? Unveiling Factual Errors in Large Language Models

TL;DR

FactChecker introduces an automated, KG-based framework to systematically uncover factual inaccuracies in large language models. By extracting triplets from Wikidata, constructing directed knowledge graphs, and generating diverse Yes-No, MC, and WH questions (including one-hop and two-hop reasoning), it enables robust, leakage-resistant evaluation of LLMs. Empirical results across six models show meaningful error triggers and demonstrate notable improvements in factual accuracy through in-context learning and fine-tuning, with manual validation confirming high error validity. The approach offers a scalable, transparent testbed for LLM evaluation and ongoing improvement, with resources publicly released for research replication and extension.

Abstract

Large Language Models (LLMs) like ChatGPT are foundational in various applications due to their extensive knowledge from pre-training and fine-tuning. Despite this, they are prone to generating factual and commonsense errors, raising concerns in critical areas like healthcare, journalism, and education to mislead users. Current methods for evaluating LLMs' veracity are limited by test data leakage or the need for extensive human labor, hindering efficient and accurate error detection. To tackle this problem, we introduce a novel, automatic testing framework, FactChecker, aimed at uncovering factual inaccuracies in LLMs. This framework involves three main steps: First, it constructs a factual knowledge graph by retrieving fact triplets from a large-scale knowledge database. Then, leveraging the knowledge graph, FactChecker employs a rule-based approach to generates three types of questions (Yes-No, Multiple-Choice, and WH questions) that involve single-hop and multi-hop relations, along with correct answers. Lastly, it assesses the LLMs' responses for accuracy using tailored matching strategies for each question type. Our extensive tests on six prominent LLMs, including text-davinci-002, text-davinci-003, ChatGPT~(gpt-3.5-turbo, gpt-4), Vicuna, and LLaMA-2, reveal that FactChecker can trigger factual errors in up to 45\% of questions in these models. Moreover, we demonstrate that FactChecker's test cases can improve LLMs' factual accuracy through in-context learning and fine-tuning (e.g., llama-2-13b-chat's accuracy increase from 35.3\% to 68.5\%). We are making all code, data, and results available for future research endeavors.

Figures (3)

• Figure 1: An illustration of the framework of FactChecker.
• Figure 2: The retrieval process for fact triplets.
• Figure 3: The proposed rule-based method for Question Generation.