Faithful Model Evaluation for Model-Based Metrics

TL;DR

This work addresses the problem that standard significance testing can be biased when evaluation uses model-based metrics, because metric-model prediction errors affect variance and confidence intervals. It introduces a mathematical framework with $\,mathrm{Var}^{M}$ to account for metric-prediction errors, and derives $p^R$ and adjusted variances, including an unbiased estimator with the Bessel correction, yielding corrected confidence intervals. The approach is validated through experiments on public toxicity benchmarks and a production NLU system, showing that incorporating metric-model errors can change conclusions in some cases. The contribution provides a principled, faithful evaluation method for model-based metrics, with practical implications for safer deployment and interpretation of NLP evaluations, and calls for extensions to non-binary metrics and relaxed IID assumptions.

Abstract

Statistical significance testing is used in natural language processing (NLP) to determine whether the results of a study or experiment are likely to be due to chance or if they reflect a genuine relationship. A key step in significance testing is the estimation of confidence interval which is a function of sample variance. Sample variance calculation is straightforward when evaluating against ground truth. However, in many cases, a metric model is often used for evaluation. For example, to compare toxicity of two large language models, a toxicity classifier is used for evaluation. Existing works usually do not consider the variance change due to metric model errors, which can lead to wrong conclusions. In this work, we establish the mathematical foundation of significance testing for model-based metrics. With experiments on public benchmark datasets and a production system, we show that considering metric model errors to calculate sample variances for model-based metrics changes the conclusions in certain experiments.