Noisy but Valid: Robust Statistical Evaluation of LLMs with Imperfect Judges

TL;DR

The paper introduces Noisy Hypothesis Testing to certify LLM reliability using imperfect judges by calibrating judge behavior (TPR and FPR) on a small human-labelled set and applying a variance-aware threshold to a large judge-labelled evaluation, guaranteeing finite-sample Type-I error control. It derives exact conditions under which noisy testing improves over direct evaluation and quantifies the Oracle Gap, then validates the framework on tasks including Jigsaw, Hate Speech, and SafeRLHF. Empirical results show Noisy HT achieves higher power than Direct HT in regimes of strong judge quality, while the Oracle baseline defines an upper bound on performance; the work also provides diagnostics on judge reliability, data sizing, and prompt design. Overall, the approach offers a scalable, interpretable, and statistically rigorous pathway to certify LLM safety at scale, with explicit trade-offs between judge quality, dataset sizes, and certification level.

Abstract

Reliable certification of Large Language Models (LLMs)-verifying that failure rates are below a safety threshold-is critical yet challenging. While "LLM-as-a-Judge" offers scalability, judge imperfections, noise, and bias can invalidate statistical guarantees. We introduce a "Noisy but Valid" hypothesis testing framework to address this. By leveraging a small human-labelled calibration set to estimate the judge's True Positive and False Positive Rates (TPR/FPR), we derive a variance-corrected critical threshold applied to a large judge-labelled dataset. Crucially, our framework theoretically guarantees finite-sample Type-I error control (validity) despite calibration uncertainty. This distinguishes our work from Prediction-Powered Inference (PPI), positioning our method as a diagnostic tool that explicitly models judge behavior rather than a black-box estimator. Our contributions include: (1) Theoretical Guarantees: We derive the exact conditions under which noisy testing yields higher statistical power than direct evaluation; (2) Empirical Validation: Experiments on Jigsaw Comment, Hate Speech and SafeRLHF confirm our theory; (3) The Oracle Gap: We reveal a significant performance gap between practical methods and the theoretical "Oracle" (perfectly known judge parameters), quantifying the cost of estimation. Specifically, we provide the first systematic treatment of the imperfect-judge setting, yielding interpretable diagnostics of judge reliability and clarifying how evaluation power depends on judge quality, dataset size, and certification levels. Together, these results sharpen understanding of statistical evaluation with LLM judges, and highlight trade-offs among competing inferential tools.

Figures (15)

• Figure 1: Performance comparison of certification procedures ($\alpha = 0.25$, $\zeta = 0.05$, $n_M = 100$, $n_J = 10,000$). (A-C) Type-I and Type-II error probabilities versus LLM failure rate threshold ($\alpha$) under varying Judge Qualities ($\text{TPR},\text{FPR}$). Solid lines represent practical methods (Direct HT, Noisy HT, PPI Variants); Dashed green lines represent the theoretical upper bound for Noisy HT (oracle $\text{TPR}$ and $\text{FPR}$). Oracle Gap: All practical methods, underperform the Oracle Noisy HT, highlighting the cost of parameter estimation. (D) Practical Guidance: Regions on the $\text{TPR}$-$\text{FPR}$ plane where our Noisy HT statistically outperforms (green) or underperforms (red) the Direct HT baseline.
• Figure 2: Overview of the Judge-Augmented LLM Certification Pipeline (Noisy HT). (A) Data Generation: The framework utilizes two datasets: a large dataset evaluated by the LLM-as-a-Judge ($\mathcal{D}_J$) and a small, high-quality human-labelled dataset ($\mathcal{D}_M$). We further construct an augmented dataset $\tilde{\mathcal{D}}_M$ by collecting judge predictions for the samples in $\mathcal{D}_M$. (B) Certification Procedure: 1) Judge Calibration: The augmented set $\tilde{\mathcal{D}}_M$ is used to estimate the judge's performance parameters ($\widehat{\text{TPR}}$ and $\widehat{\text{FPR}}$). 2) Variance-Corrected Testing: We construct a proxy hypothesis test on the large dataset $\mathcal{D}_J$. The critical threshold $c_J'$ is calculated using the estimated parameters and explicitly incorporates the variance terms from the small calibration set to guarantee statistical validity (Type-I error control). 3) Decision: The observed noisy failure rate $\hat{R}_J$ is compared against $c_J'$ to accept or reject the null hypothesis. See Algorithm \ref{['alg:noisy_hypothesis_testing_procedure']} for details; alternatively, Direct HT relies solely on $\mathcal{D}_M$ (\ref{['sec:direct_hypothesis_testing']}).
• Figure 3: Type-I and Type-II error rate of various hypothesis testing procedures for Qwen2.5-0.5B-Instruct and LLaMA-3.2-1B-Instruct toxicity classifiers coupled with a LLaMA-3.1-8B-Instruct judge on the Jigsaw Toxic Comment Classification dataset. Additional experiments with other language models and judges provided in \ref{['sup:additional_experiments']}.
• Figure 4: Type-I and Type-II error rate of various hypothesis testing procedures for Qwen2.5-0.5B-Instruct and LLaMA-3.2-1B-Instruct toxicity classifiers coupled with a LLaMA-3.1-8B-Instruct judge on the Hate Speech Offensive dataset. Additional experiments with other language models and judges provided in \ref{['sup:additional_experiments']}.
• Figure 5: Type-I and Type-II error rate of various hypothesis testing procedures for an Alpaca-7B language model coupled with LLaMA-3.1-8B-Instruct and LLaMA-3.3-70B-Instruct judges on the SafeRLHF dataset. Additional experiments with other judges provided in \ref{['sup:additional_experiments']}.

Theorems & Definitions (4)

• Theorem 5.1
• Theorem 5.2
• Theorem 5.3
• Theorem 5.4