Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Do Persona-Infused LLMs Affect Performance in a Strategic Reasoning Game?

John Licato, Stephen Steinle, Brayden Hollis

TL;DR

This work investigates whether prompting LLMs with persona descriptions translates into strategic decision-making in a Risk-inspired game (PERIL) and whether translating personas through a personality inventory improves behavioral realism. The authors compare direct heuristic prompts (DH) with a PID-5–inspired personality inventory (PI) mediator across multiple LLMs and extensive tournament play, demonstrating that PI yields stronger correlations between inferred traits and performance, and more coherent mappings between traits and heuristics. Key contributions include the PERIL platform, a principled persona-selection method, and a psychometric translation approach that enhances heuristic reliability and face validity. The findings suggest that simply prompting for a persona is insufficient to produce diverse, intent-driven decisions, and that inventory-based mediation provides a viable path to study personality effects in adversarial AI, with implications for simulations and training systems.

Abstract

Although persona prompting in large language models appears to trigger different styles of generated text, it is unclear whether these translate into measurable behavioral differences, much less whether they affect decision-making in an adversarial strategic environment that we provide as open-source. We investigate the impact of persona prompting on strategic performance in PERIL, a world-domination board game. Specifically, we compare the effectiveness of persona-derived heuristic strategies to those chosen manually. Our findings reveal that certain personas associated with strategic thinking improve game performance, but only when a mediator is used to translate personas into heuristic values. We introduce this mediator as a structured translation process, inspired by exploratory factor analysis, that maps LLM-generated inventory responses into heuristics. Results indicate our method enhances heuristic reliability and face validity compared to directly inferred heuristics, allowing us to better study the effect of persona types on decision making. These insights advance our understanding of how persona prompting influences LLM-based decision-making and propose a heuristic generation method that applies psychometric principles to LLMs.

Do Persona-Infused LLMs Affect Performance in a Strategic Reasoning Game?

TL;DR

This work investigates whether prompting LLMs with persona descriptions translates into strategic decision-making in a Risk-inspired game (PERIL) and whether translating personas through a personality inventory improves behavioral realism. The authors compare direct heuristic prompts (DH) with a PID-5–inspired personality inventory (PI) mediator across multiple LLMs and extensive tournament play, demonstrating that PI yields stronger correlations between inferred traits and performance, and more coherent mappings between traits and heuristics. Key contributions include the PERIL platform, a principled persona-selection method, and a psychometric translation approach that enhances heuristic reliability and face validity. The findings suggest that simply prompting for a persona is insufficient to produce diverse, intent-driven decisions, and that inventory-based mediation provides a viable path to study personality effects in adversarial AI, with implications for simulations and training systems.

Abstract

Although persona prompting in large language models appears to trigger different styles of generated text, it is unclear whether these translate into measurable behavioral differences, much less whether they affect decision-making in an adversarial strategic environment that we provide as open-source. We investigate the impact of persona prompting on strategic performance in PERIL, a world-domination board game. Specifically, we compare the effectiveness of persona-derived heuristic strategies to those chosen manually. Our findings reveal that certain personas associated with strategic thinking improve game performance, but only when a mediator is used to translate personas into heuristic values. We introduce this mediator as a structured translation process, inspired by exploratory factor analysis, that maps LLM-generated inventory responses into heuristics. Results indicate our method enhances heuristic reliability and face validity compared to directly inferred heuristics, allowing us to better study the effect of persona types on decision making. These insights advance our understanding of how persona prompting influences LLM-based decision-making and propose a heuristic generation method that applies psychometric principles to LLMs.

Paper Structure

This paper contains 28 sections, 3 equations, 16 figures, 35 tables.

Table of Contents

  1. Introduction
  2. Novel Contributions and Summary of Findings
  3. Related Work
  4. Experimental Setup
  5. Game Environment
  6. Persona Selection
  7. Heuristic Selection
  8. Tournament Games
  9. Results
  10. Comparison of Personality Inventory Players
  11. Comparison of Personality Inventory and Direct Heuristic Players
  12. Conclusion
  13. Future Work.
  14. Limitations.
  15. Appendix
  16. ...and 13 more sections

Figures (16)

  • Figure 1: PERIL, our implementation inspired by a popular strategic world conquest board game, was used to study the effects of persona-based prompting in LLMs.
  • Figure 2: Pipeline from persona descriptions to heuristic agents. Note that LLMs generate heuristic proxies (DH or PI), but do not play PERIL directly.
  • Figure 3: Heuristic Correlations - DH1 - GPT4
  • Figure 4: Heuristic Correlations - PI1 - GPT4
  • Figure 5: Heuristic Correlations - DH1 - Mistral Small
  • ...and 11 more figures