Digital Red Queen: Adversarial Program Evolution in Core War with LLMs

TL;DR

DRQ addresses the limitation of static optimization in LLM-driven evolution by introducing a dynamic, Red Queen–style self-play loop in Core War. It combines MAP-Elites with an LLM-based mutation operator to generate a lineage of warriors that continually adapts to defeat an expanding history of opponents, yielding increasingly general and robust strategies. The study reveals convergent phenotypes across independent runs while maintaining genotypic diversity, and shows that Core War is a valuable sandbox for analyzing adversarial adaptation and for benchmarking LLM-guided evolution approaches. The results highlight the potential of dynamic objectives to produce transferable insights for real-world domains like cybersecurity and drug resistance, while underscoring the utility of Core War as a controllable testbed for open-ended evolution with LLMs.

Abstract

Large language models (LLMs) are increasingly being used to evolve solutions to problems in many domains, in a process inspired by biological evolution. However, unlike biological evolution, most LLM-evolution frameworks are formulated as static optimization problems, overlooking the open-ended adversarial dynamics that characterize real-world evolutionary processes. Here, we study Digital Red Queen (DRQ), a simple self-play algorithm that embraces these so-called "Red Queen" dynamics via continual adaptation to a changing objective. DRQ uses an LLM to evolve assembly-like programs, called warriors, which compete against each other for control of a virtual machine in the game of Core War, a Turing-complete environment studied in artificial life and connected to cybersecurity. In each round of DRQ, the model evolves a new warrior to defeat all previous ones, producing a sequence of adapted warriors. Over many rounds, we observe that warriors become increasingly general (relative to a set of held-out human warriors). Interestingly, warriors also become less behaviorally diverse across independent runs, indicating a convergence pressure toward a general-purpose behavioral strategy, much like convergent evolution in nature. This result highlights a potential value of shifting from static objectives to dynamic Red Queen objectives. Our work positions Core War as a rich, controllable sandbox for studying adversarial adaptation in artificial systems and for evaluating LLM-based evolution methods. More broadly, the simplicity and effectiveness of DRQ suggest that similarly minimal self-play approaches could prove useful in other more practical multi-agent adversarial domains, like real-world cybersecurity or combating drug resistance.

Figures (12)

• Figure 1: Static optimization baseline. Static optimization (single-round DRQ) with an LLM can discover specialist warriors that collectively match or surpass $96.3\%$ of $294$ human-designed warriors, far above the LLM's zero-shotting and best-of-$N$ baselines. However, individual warriors are brittle, defeating or matching only $27.9\%$ of human-designed warriors on average.
• Figure 2: DRQ warriors are statistically converging toward a single general-purpose behavior over rounds. Each point in all plots is computed from 96 independent DRQ runs with different initial warriors. Logarithmic or linear models are fit to the data, and reported $p$-values test the null hypothesis that the slope of the fitted model is zero. $K$ is the history length in DRQ. Left: The warriors' average generality increases over rounds. Generality is defined as the fraction of unseen human warriors defeated or tied, measuring a warrior’s ability to adapt to novel threats in a zero-shot setting. Center Left: The variance of the warriors’ phenotype across independent DRQ runs decreases over rounds. A warrior’s phenotype is defined as a vector of fitness values against each unseen human warrior. Center Right: The rate of change of the phenotype decreases over rounds. Under the log model, full convergence would require an exponential number of rounds. Right: The variance of the warriors’ genotype across independent DRQ runs remains static over rounds. A warrior’s genotype is defined as a text embedding of its source code.
• Figure 3: Convergence is observed in the phenotype but not in the genotype. Moreover, this convergence pressure is relatively weak and does not appear in every DRQ run, but only emerges statistically when aggregating many independent runs.
• Figure 4: Cyclic behavior in DRQ across champions from different rounds. Arrows show cycles of three warriors that have a rock-paper-scissors dynamic. DRQ with $K=1$ exhibits many cycles, whereas full DRQ reduces them.
• Figure 5: MAP-Elites archive with fitness averaged across rounds and runs. The axes correspond to memory coverage (total number of unique addresses accessed during execution) and spawned threads (total number of threads launched via a fork opcode) The best warriors have low memory coverage and many spawned threads.