Large Language Models as Pokémon Battle Agents: Strategic Play and Content Generation

TL;DR

The paper investigates whether large language models can act as competent turn-based Pokémon battle agents, capable of both tactical decision-making and novel move content generation, without domain-specific training. It introduces a deterministic battle engine and a dual-evaluation pipeline (mechanical validity and LLM-based creativity) to assess moves, plus an LLM-vs-LLM framework to compare multiple models. Across eight experiments, the study finds meaningful strategic competence in LLMs, with clear trade-offs between reasoning depth (chain-of-thought) and latency, and with model-specific strengths in speed, balance, or creativity. The work demonstrates the viability of LLMs as both opponents and content designers in interactive entertainment, pointing to practical design implications for adaptive difficulty and procedurally generated content.

Abstract

Strategic decision-making in Pokémon battles presents a unique testbed for evaluating large language models. Pokémon battles demand reasoning about type matchups, statistical trade-offs, and risk assessment, skills that mirror human strategic thinking. This work examines whether Large Language Models (LLMs) can serve as competent battle agents, capable of both making tactically sound decisions and generating novel, balanced game content. We developed a turn-based Pokémon battle system where LLMs select moves based on battle state rather than pre-programmed logic. The framework captures essential Pokémon mechanics: type effectiveness multipliers, stat-based damage calculations, and multi-Pokémon team management. Through systematic evaluation across multiple model architectures we measured win rates, decision latency, type-alignment accuracy, and token efficiency. These results suggest LLMs can function as dynamic game opponents without domain-specific training, offering a practical alternative to reinforcement learning for turn-based strategic games. The dual capability of tactical reasoning and content creation, positions LLMs as both players and designers, with implications for procedural generation and adaptive difficulty systems in interactive entertainment.