SaGE: Evaluating Moral Consistency in Large Language Models

TL;DR

An information-theoretic measure called Semantic Graph Entropy (SaGE), grounded in the concept of “Rules of Thumb” (RoTs) to measure a model’s moral consistency is proposed, which reveals that task accuracy and consistency are independent problems, and there is a dire need to investigate these issues further.

Abstract

Despite recent advancements showcasing the impressive capabilities of Large Language Models (LLMs) in conversational systems, we show that even state-of-the-art LLMs are morally inconsistent in their generations, questioning their reliability (and trustworthiness in general). Prior works in LLM evaluation focus on developing ground-truth data to measure accuracy on specific tasks. However, for moral scenarios that often lack universally agreed-upon answers, consistency in model responses becomes crucial for their reliability. To address this issue, we propose an information-theoretic measure called Semantic Graph Entropy (SaGE), grounded in the concept of "Rules of Thumb" (RoTs) to measure a model's moral consistency. RoTs are abstract principles learned by a model and can help explain their decision-making strategies effectively. To this extent, we construct the Moral Consistency Corpus (MCC), containing 50K moral questions, responses to them by LLMs, and the RoTs that these models followed. Furthermore, to illustrate the generalizability of SaGE, we use it to investigate LLM consistency on two popular datasets -- TruthfulQA and HellaSwag. Our results reveal that task-accuracy and consistency are independent problems, and there is a dire need to investigate these issues further.

Figures (4)

• Figure 1: An example of GPT-3.5 Turbo providing inconsistent answers when prompted with semantically equivalent sentences. The responses were recorded through the OpenAI API via zero-shot prompting, on September $\text{20}^\text{th}$, 2023. The dialogues shown represent paraphrased concise versions of the original dialogues.
• Figure 2: An illustration of our pipeline to evaluate moral consistency. Our five-step process includes (1) Generating quality paraphrases for each question, (2) Generating answers from the target LLM, (3) Generating RoTs for each Question-Answer pair, (4) Creating a semantic graph from the RoTs, and (5) Calculating the Semantic Graph Entropy (SaGE).
• Figure 3: Representation of the variation in ROUGE and SaGE scores across different temperatures. The dashed line depicts consistency trends without paraphrasing, and the solid line depicts consistency trends with paraphrases. The figure reveals that consistency is not dependent on temperature.
• Figure 4: Scatter plot between SaGE scores and dataset's task accuracies. We observe no significant correlation, implying that consistency and accuracy are two different problems.