Measuring Reliability of Large Language Models through Semantic Consistency

TL;DR

The paper tackles the reliability of large language models by measuring semantic consistency across paraphrased prompts. It introduces a general framework, Cons_sem, that replaces exact lexical matching with semantic agreement functions to evaluate open-ended text generation. By testing multiple agreement functions (paraphrase detection, entailment, BERTScore) across model scales and decoding methods on TruthfulQA paraphrases, it shows that semantic consistency better tracks human judgments than lexical similarity, while decoding strategy largely drives inconsistencies. These findings highlight the orthogonality of consistency and accuracy and offer a reproducible approach for evaluating and improving trustworthy NLG systems.

Abstract

While large pretrained language models (PLMs) demonstrate incredible fluency and performance on many natural language tasks, recent work has shown that well-performing PLMs are very sensitive to what prompts are feed into them. Even when prompts are semantically identical, language models may give very different answers. When considering safe and trustworthy deployments of PLMs we would like their outputs to be consistent under prompts that mean the same thing or convey the same intent. While some work has looked into how state-of-the-art PLMs address this need, they have been limited to only evaluating lexical equality of single- or multi-word answers and do not address consistency of generative text sequences. In order to understand consistency of PLMs under text generation settings, we develop a measure of semantic consistency that allows the comparison of open-ended text outputs. We implement several versions of this consistency metric to evaluate the performance of a number of PLMs on paraphrased versions of questions in the TruthfulQA dataset, we find that our proposed metrics are considerably more consistent than traditional metrics embodying lexical consistency, and also correlate with human evaluation of output consistency to a higher degree.