Beyond One World: Benchmarking Super Heros in Role-Playing Across Multiversal Contexts

TL;DR

Beyond One World introduces a multiversal character-roleplay benchmark that assesses version-specific portrayal across 30 heroes and 90 canon variants via two tasks: Canon Events and Moral Dilemmas. The authors separate internal thinking from outward acting and propose Think-Act Matching to quantify alignment and trustworthiness, evaluating both reasoning-oriented and standard LLMs. Key findings show that chain-of-thought prompting yields mixed results, cross-version generalization remains difficult, and models often specialize in either thinking or acting rather than both. The work provides a challenging, fine-grained evaluation framework with a large, expert-curated dataset to advance trustworthy, contextually coherent character role-play in LLMs.

Abstract

Large language models (LLMs) are increasingly used as role-playing agents, yet their capacity to faithfully and consistently portray version-specific characters -- for example, superheroes across comic and cinematic universes -- remains underexplored. Superhero canons such as Marvel and DC provide a rich testbed: decades of storytelling yield multiple incarnations of the same character with distinct histories, values, and moral codes. To study this problem, we introduce Beyond One World, a benchmark for character-grounded roleplay spanning 30 iconic heroes and 90 canon-specific versions. The benchmark comprises two tasks: (i) Canon Events, which probes factual recall of pivotal life stages, and (ii) Moral Dilemmas, which confronts models with ethically charged scenarios. We score responses for canonical accuracy and reasoning fidelity under a framework that separates internal deliberation ("thinking") from outward decisions ("acting"). We further propose Think-Act Matching, a metric that quantifies alignment between reasons and actions and serves as a proxy for model trustworthiness. Experiments across reasoning- and non-reasoning-oriented models yield three findings: (1) chain-of-thought prompting improves narrative coherence in weaker models but can reduce canonical accuracy in stronger ones; (2) cross-version generalization within a character remains a major obstacle; and (3) models often excel at either thinking or acting, but rarely both. Beyond One World exposes critical gaps in multiversal consistency and reasoning alignment, offering a challenging evaluation for role-playing LLMs.

Figures (9)

• Figure 1: Left image is inferencing llm that prompted hero persona to do canon event task, and the right image is prompted llm to do dilemma situation task.
• Figure 2: A pipeline for reasoning analysis, after getting output from task Canon and Dilemma the output response are structured into $<$thinking$>$ and $<$acting$>$ by Gpt4o-mini judged by Sonnet3.7 that was prompted to be a judge with attributes of that character.
• Figure 3: Accuracy result from cross characters evaluation on task canon and task dilemma.
• Figure 4: scores for acting-thinking.
• Figure 5: Barplot of our benchmark tasks.The Blue one consist of Childhood,Pre-Hero and Hero question type,the Red one is balanced Dilemma situation.