Beyond Single-Sentence Prompts: Upgrading Value Alignment Benchmarks with Dialogues and Stories

TL;DR

This work targets the inadequacy of single-sentence prompts for assessing large language model value alignment and introduces C-Plus Values, a Chinese benchmark built on adversarial, context-rich formats. It converts expert questions from CVALUES into negative viewpoints and expands them into two formats—multi-turn dialogues and story scenarios—guided by Cognitive Load Theory to stress reasoning about responsibilities and biases. The authors implement both manual and automatic evaluation pipelines, demonstrating that the upgraded benchmark more effectively reveals latent biases and judgment gaps across models, with dialogue tasks generally easier and story tasks revealing deeper ethical reasoning challenges. Their findings argue for context-aware, adversarial testing in safety assessments and present a scalable framework for evaluating Chinese LLMs, while acknowledging limitations in cultural scope and evaluation subjectivity.

Abstract

Evaluating the value alignment of large language models (LLMs) has traditionally relied on single-sentence adversarial prompts, which directly probe models with ethically sensitive or controversial questions. However, with the rapid advancements in AI safety techniques, models have become increasingly adept at circumventing these straightforward tests, limiting their effectiveness in revealing underlying biases and ethical stances. To address this limitation, we propose an upgraded value alignment benchmark that moves beyond single-sentence prompts by incorporating multi-turn dialogues and narrative-based scenarios. This approach enhances the stealth and adversarial nature of the evaluation, making it more robust against superficial safeguards implemented in modern LLMs. We design and implement a dataset that includes conversational traps and ethically ambiguous storytelling, systematically assessing LLMs' responses in more nuanced and context-rich settings. Experimental results demonstrate that this enhanced methodology can effectively expose latent biases that remain undetected in traditional single-shot evaluations. Our findings highlight the necessity of contextual and dynamic testing for value alignment in LLMs, paving the way for more sophisticated and realistic assessments of AI ethics and safety.

Figures (13)

• Figure 1: The comparison between C-Plus Values and Cvalues.
• Figure 2: The C-Plus Values flowchart.
• Figure 3: The prompts for generating multi-turn dialogues.
• Figure 4: The prompts for generating story scenarios.
• Figure 5: Thematic Similarity Between Dialogues (Left) and Stories (Right).