Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Right vs. Right: Can LLMs Make Tough Choices?

Jiaqing Yuan, Pradeep K. Murukannaiah, Munindar P. Singh

TL;DR

This study probes how large language models navigate ethical dilemmas where both options are valuable, using a 1,730-dilemma corpus grounded in Kidder's four value pairs. It analyzes sensitivity to prompt formulations, consistency of moral preferences, and the role of consequences, comparing explicit value guidance with exemplar-based prompts across 20 LLMs. Key findings show pronounced value biases, deontological leanings in larger models, and that explicit prompts more reliably steer moral choices than few-shot exemplars, with significant variation across value pairs. The work informs the design of alignment strategies and safe deployment of LLMs in ethically sensitive contexts, while acknowledging limitations related to prompt sensitivity and framework scope.

Abstract

An ethical dilemma describes a choice between two "right" options involving conflicting moral values. We present a comprehensive evaluation of how LLMs navigate ethical dilemmas. Specifically, we investigate LLMs on their (1) sensitivity in comprehending ethical dilemmas, (2) consistency in moral value choice, (3) consideration of consequences, and (4) ability to align their responses to a moral value preference explicitly or implicitly specified in a prompt. Drawing inspiration from a leading ethical framework, we construct a dataset comprising 1,730 ethical dilemmas involving four pairs of conflicting values. We evaluate 20 well-known LLMs from six families. Our experiments reveal that: (1) LLMs exhibit pronounced preferences between major value pairs, and prioritize truth over loyalty, community over individual, and long-term over short-term considerations. (2) The larger LLMs tend to support a deontological perspective, maintaining their choices of actions even when negative consequences are specified. (3) Explicit guidelines are more effective in guiding LLMs' moral choice than in-context examples. Lastly, our experiments highlight the limitation of LLMs in comprehending different formulations of ethical dilemmas.

Right vs. Right: Can LLMs Make Tough Choices?

TL;DR

This study probes how large language models navigate ethical dilemmas where both options are valuable, using a 1,730-dilemma corpus grounded in Kidder's four value pairs. It analyzes sensitivity to prompt formulations, consistency of moral preferences, and the role of consequences, comparing explicit value guidance with exemplar-based prompts across 20 LLMs. Key findings show pronounced value biases, deontological leanings in larger models, and that explicit prompts more reliably steer moral choices than few-shot exemplars, with significant variation across value pairs. The work informs the design of alignment strategies and safe deployment of LLMs in ethically sensitive contexts, while acknowledging limitations related to prompt sensitivity and framework scope.

Abstract

An ethical dilemma describes a choice between two "right" options involving conflicting moral values. We present a comprehensive evaluation of how LLMs navigate ethical dilemmas. Specifically, we investigate LLMs on their (1) sensitivity in comprehending ethical dilemmas, (2) consistency in moral value choice, (3) consideration of consequences, and (4) ability to align their responses to a moral value preference explicitly or implicitly specified in a prompt. Drawing inspiration from a leading ethical framework, we construct a dataset comprising 1,730 ethical dilemmas involving four pairs of conflicting values. We evaluate 20 well-known LLMs from six families. Our experiments reveal that: (1) LLMs exhibit pronounced preferences between major value pairs, and prioritize truth over loyalty, community over individual, and long-term over short-term considerations. (2) The larger LLMs tend to support a deontological perspective, maintaining their choices of actions even when negative consequences are specified. (3) Explicit guidelines are more effective in guiding LLMs' moral choice than in-context examples. Lastly, our experiments highlight the limitation of LLMs in comprehending different formulations of ethical dilemmas.
Paper Structure (30 sections, 6 figures, 9 tables)

This paper contains 30 sections, 6 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Contributions
  3. Organization
  4. Related Work
  5. Morality in LLMs
  6. Value Alignment
  7. The Ethical Dilemma Corpus
  8. Ethical Theory
  9. Dataset Construction
  10. Example: Truth vs. Loyalty Dilemma
  11. Experiments
  12. Task Formulation
  13. Moral value queries
  14. Consequences of actions
  15. Preference handling
  16. ...and 15 more sections

Figures (6)

  • Figure 1: Moral choice agreement for different prompts. The higher the agreement the better the task comprehension.
  • Figure 2: Moral value preference query results for each conflicting value pair.
  • Figure 3: Percentage of flipping the choice when consequences are altered, e.g., an LLM switches from "Action A" to "Action B" when a negative outcome is added to "Action A" and a positive outcome is added to "Action B".
  • Figure 4: Standard deviation of the percentage of choosing the first action across four consequence-related prompts.
  • Figure 5: Percentage of flipping the baseline choice when the prompt states opposite preference, e.g., if the baseline action is Action A, the model changes its choice to Action B when the prompt states preference for the second value.
  • ...and 1 more figures