Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

LLM Ethics Benchmark: A Three-Dimensional Assessment System for Evaluating Moral Reasoning in Large Language Models

Junfeng Jiao, Saleh Afroogh, Abhejay Murali, Kevin Chen, David Atkinson, Amit Dhurandhar

TL;DR

This work introduces a three-dimensional LLM Ethics Benchmark comprising Moral Foundation Alignment (MFA), a Reasoning Quality Index (RQI), and Value Consistency Assessment (VCA) to quantify moral reasoning. It adapts established human-morality instruments—MFQ-30, World Values Survey, and Moral Dilemmas—to standardized prompts with a ground-truth, multi-metric evaluation framework that includes semantic reasoning analysis and cross-model consistency. Experimental results show leading models achieving ~90 composite scores but reveal stronger performance on basic moral foundations than on complex dilemma resolution, with cultural biases and inconsistency as notable failure modes. The framework is open-source and designed to guide model development, benchmarking, and policy discussions, providing practical paths toward more ethically aligned and culturally sensitive AI systems.

Abstract

This study establishes a novel framework for systematically evaluating the moral reasoning capabilities of large language models (LLMs) as they increasingly integrate into critical societal domains. Current assessment methodologies lack the precision needed to evaluate nuanced ethical decision-making in AI systems, creating significant accountability gaps. Our framework addresses this challenge by quantifying alignment with human ethical standards through three dimensions: foundational moral principles, reasoning robustness, and value consistency across diverse scenarios. This approach enables precise identification of ethical strengths and weaknesses in LLMs, facilitating targeted improvements and stronger alignment with societal values. To promote transparency and collaborative advancement in ethical AI development, we are publicly releasing both our benchmark datasets and evaluation codebase at https://github.com/ The-Responsible-AI-Initiative/LLM_Ethics_Benchmark.git.

LLM Ethics Benchmark: A Three-Dimensional Assessment System for Evaluating Moral Reasoning in Large Language Models

TL;DR

This work introduces a three-dimensional LLM Ethics Benchmark comprising Moral Foundation Alignment (MFA), a Reasoning Quality Index (RQI), and Value Consistency Assessment (VCA) to quantify moral reasoning. It adapts established human-morality instruments—MFQ-30, World Values Survey, and Moral Dilemmas—to standardized prompts with a ground-truth, multi-metric evaluation framework that includes semantic reasoning analysis and cross-model consistency. Experimental results show leading models achieving ~90 composite scores but reveal stronger performance on basic moral foundations than on complex dilemma resolution, with cultural biases and inconsistency as notable failure modes. The framework is open-source and designed to guide model development, benchmarking, and policy discussions, providing practical paths toward more ethically aligned and culturally sensitive AI systems.

Abstract

This study establishes a novel framework for systematically evaluating the moral reasoning capabilities of large language models (LLMs) as they increasingly integrate into critical societal domains. Current assessment methodologies lack the precision needed to evaluate nuanced ethical decision-making in AI systems, creating significant accountability gaps. Our framework addresses this challenge by quantifying alignment with human ethical standards through three dimensions: foundational moral principles, reasoning robustness, and value consistency across diverse scenarios. This approach enables precise identification of ethical strengths and weaknesses in LLMs, facilitating targeted improvements and stronger alignment with societal values. To promote transparency and collaborative advancement in ethical AI development, we are publicly releasing both our benchmark datasets and evaluation codebase at https://github.com/ The-Responsible-AI-Initiative/LLM_Ethics_Benchmark.git.
Paper Structure (25 sections, 3 equations, 7 figures, 4 tables)

This paper contains 25 sections, 3 equations, 7 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Background
  3. Research Gap
  4. Objectives
  5. Paper Structure
  6. Related Work
  7. AI Evaluation Techniques
  8. Human Morality and Ethics Literature
  9. Customizing Moral Evaluation for LLMs
  10. Selection and Adaptation of Human Moral Tests
  11. Technical Implementation and Prompt Engineering
  12. Proposed Methodology for Testing LLM Moral Reasoning
  13. Experimental Setup
  14. Implications for Developers and Stakeholders
  15. Challenges and Limitations
  16. ...and 10 more sections

Figures (7)

  • Figure 1: The need for integrated research bridging technical evaluation and human morality
  • Figure 2: Technical implementation workflow for moral reasoning assessment in LLMs
  • Figure 3: Moral foundation alignment across models compared with human baseline
  • Figure 4: Distribution of reasoning depth scores in moral dilemma responses
  • Figure 5: Consistency scores across evaluation rounds with prompt variations
  • ...and 2 more figures