Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Crisis-Bench: Benchmarking Strategic Ambiguity and Reputation Management in Large Language Models

Cooper Lin, Maohao Ran, Yanting Zhang, Zhenglin Wan, Hongwei Fan, Yibo Xu, Yike Guo, Wei Xue, Jun Song

TL;DR

Crisis-Bench tackles the gap between universal safety alignment and professional utility by introducing a dynamic, multi-agent crisis simulation that models information asymmetry between private company knowledge and public knowledge. The framework uses a Dual-Knowledge Architecture and an Adjudicator-Market Loop to translate qualitative reputation management into quantitative economic signals, enabling rigorous evaluation of strategic withholding and narrative control. Experimental results reveal an Alignment Tax: rigid safety alignment can hinder complex professional tasks, with larger, more capable models achieving better Theory of Mind and more effective crisis control, while overemphasis on transparency can exacerbate crisis severity and financial cost. The work advocates for context-aware alignment that differentiates fiduciary responsibility from harmlessness and provides a robust baseline for developing professional, strategically-aware LLM agents.

Abstract

Standard safety alignment optimizes Large Language Models (LLMs) for universal helpfulness and honesty, effectively instilling a rigid "Boy Scout" morality. While robust for general-purpose assistants, this one-size-fits-all ethical framework imposes a "transparency tax" on professional domains requiring strategic ambiguity and information withholding, such as public relations, negotiation, and crisis management. To measure this gap between general safety and professional utility, we introduce Crisis-Bench, a multi-agent Partially Observable Markov Decision Process (POMDP) that evaluates LLMs in high-stakes corporate crises. Spanning 80 diverse storylines across 8 industries, Crisis-Bench tasks an LLM-based Public Relations (PR) Agent with navigating a dynamic 7-day corporate crisis simulation while managing strictly separated Private and Public narrative states to enforce rigorous information asymmetry. Unlike traditional benchmarks that rely on static ground truths, we introduce the Adjudicator-Market Loop: a novel evaluation metric where public sentiment is adjudicated and translated into a simulated stock price, creating a realistic economic incentive structure. Our results expose a critical dichotomy: while some models capitulate to ethical concerns, others demonstrate the capacity for Machiavellian, legitimate strategic withholding in order to stabilize the simulated stock price. Crisis-Bench provides the first quantitative framework for assessing "Reputation Management" capabilities, arguing for a shift from rigid moral absolutism to context-aware professional alignment.

Crisis-Bench: Benchmarking Strategic Ambiguity and Reputation Management in Large Language Models

TL;DR

Crisis-Bench tackles the gap between universal safety alignment and professional utility by introducing a dynamic, multi-agent crisis simulation that models information asymmetry between private company knowledge and public knowledge. The framework uses a Dual-Knowledge Architecture and an Adjudicator-Market Loop to translate qualitative reputation management into quantitative economic signals, enabling rigorous evaluation of strategic withholding and narrative control. Experimental results reveal an Alignment Tax: rigid safety alignment can hinder complex professional tasks, with larger, more capable models achieving better Theory of Mind and more effective crisis control, while overemphasis on transparency can exacerbate crisis severity and financial cost. The work advocates for context-aware alignment that differentiates fiduciary responsibility from harmlessness and provides a robust baseline for developing professional, strategically-aware LLM agents.

Abstract

Standard safety alignment optimizes Large Language Models (LLMs) for universal helpfulness and honesty, effectively instilling a rigid "Boy Scout" morality. While robust for general-purpose assistants, this one-size-fits-all ethical framework imposes a "transparency tax" on professional domains requiring strategic ambiguity and information withholding, such as public relations, negotiation, and crisis management. To measure this gap between general safety and professional utility, we introduce Crisis-Bench, a multi-agent Partially Observable Markov Decision Process (POMDP) that evaluates LLMs in high-stakes corporate crises. Spanning 80 diverse storylines across 8 industries, Crisis-Bench tasks an LLM-based Public Relations (PR) Agent with navigating a dynamic 7-day corporate crisis simulation while managing strictly separated Private and Public narrative states to enforce rigorous information asymmetry. Unlike traditional benchmarks that rely on static ground truths, we introduce the Adjudicator-Market Loop: a novel evaluation metric where public sentiment is adjudicated and translated into a simulated stock price, creating a realistic economic incentive structure. Our results expose a critical dichotomy: while some models capitulate to ethical concerns, others demonstrate the capacity for Machiavellian, legitimate strategic withholding in order to stabilize the simulated stock price. Crisis-Bench provides the first quantitative framework for assessing "Reputation Management" capabilities, arguing for a shift from rigid moral absolutism to context-aware professional alignment.
Paper Structure (16 sections, 3 equations, 3 figures, 3 tables)

This paper contains 16 sections, 3 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Crisis-Bench
  3. Simulation Environment and State Space
  4. The Router Agent and Event Dynamics
  5. The PR Agent (Policy)
  6. Evaluation and Metrics
  7. Experiment
  8. Experimental Setup
  9. Results
  10. Related Works
  11. Conclusion
  12. Ethical Considerations
  13. Storyline Design
  14. Agent Design
  15. Evaluation Configuration
  16. ...and 1 more sections

Figures (3)

  • Figure 1: The Crisis-Bench workflow. The Router selects an event which updates the Private ($K_{priv}$) and Public ($K_{pub}$) knowledge bases. The PR Agent responds to these events, and the Adjudicator's score drives a simulated stock price, which the agent must stabilize to maximize shareholder value.
  • Figure 2: Simulation assets. The Dossier ($D$) contains the immutable ground truth. $K_{priv}$ and $K_{pub}$ maintain dynamic information asymmetry between the firm and the public, while the Event Pool ($\mathcal{E}$) drives state transitions between them.
  • Figure 3: Visualization of the simulated trust score (left) and stock price (right) on Crisis-Bench. The x-axis represents the seven simulation rounds. The y-axis is the average value of the metrics over all crisis storylines.