LLMAuditor: A Framework for Auditing Large Language Models Using Human-in-the-Loop

TL;DR

LLMAuditor introduces a general-purpose, human-in-the-loop auditing framework for large language models that operates in two phases: (1) Probe Generation, where an auditing LLM constructs and validates a diverse set of probes via a structured codebook and iterative template refinement, and (2) Probes Answering, where a separate, audited LLM responds to these probes to reveal inconsistencies, bias, or hallucinations. The probe template is rigorously developed and validated through multiple annotation rounds to ensure relevance, diversity, and inter-annotator agreement, reducing subjectivity. A case study using TruthfulQA demonstrates that LLMAuditor can generate high-quality probes from one LLM to audit another across multiple models, and its outputs are benchmarked against Adatest++ using multiple similarity and truthfulness metrics. The framework emphasizes verifiability, transparency, and generalizability, and lays groundwork for scalable auditing of unknown unknowns in LLM behavior, while acknowledging ethical considerations and current scope limitations.

Abstract

As Large Language Models (LLMs) become more pervasive across various users and scenarios, identifying potential issues when using these models becomes essential. Examples of such issues include: bias, inconsistencies, and hallucination. Although auditing the LLM for these problems is often warranted, such a process is neither easy nor accessible for most. An effective method is to probe the LLM using different versions of the same question. This could expose inconsistencies in its knowledge or operation, indicating potential for bias or hallucination. However, to operationalize this auditing method at scale, we need an approach to create those probes reliably and automatically. In this paper we propose the LLMAuditor framework which is an automatic, and scalable solution, where one uses a different LLM along with human-in-the-loop (HIL). This approach offers verifiability and transparency, while avoiding circular reliance on the same LLM, and increasing scientific rigor and generalizability. Specifically, LLMAuditor includes two phases of verification using humans: standardized evaluation criteria to verify responses, and a structured prompt template to generate desired probes. A case study using questions from the TruthfulQA dataset demonstrates that we can generate a reliable set of probes from one LLM that can be used to audit inconsistencies in a different LLM. This process is enhanced by our structured prompt template with HIL, which not only boosts the reliability of our approach in auditing but also yields the delivery of less hallucinated results. The novelty of our research stems from the development of a comprehensive, general-purpose framework that includes a HIL verified prompt template for auditing responses generated by LLMs.

Figures (2)

• Figure 1: The proposed auditing framework with two phases: (1) Probes Generation by LLM1, (2) Probes Answering by LLM2.
• Figure 2: Probe Generation Flowchart with two steps: (1) Annotator Codebook; (2) Probe Template Improvements.