Are LLMs complicated ethical dilemma analyzers?

TL;DR

The paper asks whether LLMs can emulate human ethical reasoning by assembling a structured benchmark of 196 real-world dilemmas with expert references and a multi-section prompt format. It introduces a composite evaluation framework combining lexical, syntactic, and semantic similarity, weighted via an inversion-based ranking and Analytic Hierarchy Process, and compares four frontier LLMs against non-expert baselines. Results show LLMs generally outperform non-experts in lexical/structural alignment, with GPT-4o-mini emerging as the most consistent performer, yet all models struggle with historical grounding and nuanced resolution strategies, suggesting gaps in contextual abstraction and normative reasoning. The work contributes a practical evaluation pipeline and dataset for studying AI moral reasoning and points to directions for fine-tuning and multi-agent collaboration to better align AI ethics with human norms.

Abstract

One open question in the study of Large Language Models (LLMs) is whether they can emulate human ethical reasoning and act as believable proxies for human judgment. To investigate this, we introduce a benchmark dataset comprising 196 real-world ethical dilemmas and expert opinions, each segmented into five structured components: Introduction, Key Factors, Historical Theoretical Perspectives, Resolution Strategies, and Key Takeaways. We also collect non-expert human responses for comparison, limited to the Key Factors section due to their brevity. We evaluate multiple frontier LLMs (GPT-4o-mini, Claude-3.5-Sonnet, Deepseek-V3, Gemini-1.5-Flash) using a composite metric framework based on BLEU, Damerau-Levenshtein distance, TF-IDF cosine similarity, and Universal Sentence Encoder similarity. Metric weights are computed through an inversion-based ranking alignment and pairwise AHP analysis, enabling fine-grained comparison of model outputs to expert responses. Our results show that LLMs generally outperform non-expert humans in lexical and structural alignment, with GPT-4o-mini performing most consistently across all sections. However, all models struggle with historical grounding and proposing nuanced resolution strategies, which require contextual abstraction. Human responses, while less structured, occasionally achieve comparable semantic similarity, suggesting intuitive moral reasoning. These findings highlight both the strengths and current limitations of LLMs in ethical decision-making.

Figures (4)

• Figure 1: Overview of the proposed evaluation pipeline for assessing the quality of LLMs responses to ethical dilemmas. Initially, ethical dilemma descriptions are retrieved and structured via a unified prompting framework applied uniformly across multiple LLMs and human respondents. Expert-generated outputs are processed into five distinct sections—Introduction, Key Factors in Consideration, Historical & Theoretical Perspectives, Proposed Resolution Strategies, and Key Takeaways—whereas non-expert human answers are represented only by the Key Factors in Consideration section due to their comparatively limited context. Subsequently, each LLM-generated response is quantitatively evaluated against expert opinion summaries processed independently by four distinct reference LLMs, employing multiple linguistic and semantic similarity metrics. Each metric is assigned structured and categorized weights, culminating in an aggregated final evaluation score.
• Figure 2: Final Score Distribution by Model
• Figure 3: Average Score per Metric
• Figure 4: Average Score per Metric for non-expert human scores (Left) of all criteria. Model scores (Right) are the performances of LLMs on Key Factor only. processed_1/2/3/4 are four different non-expert individual human data providers.