Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Merge and Conquer: Evolutionarily Optimizing AI for 2048

Maggie Bai, Ava Kim Cohen, Eleanor Koss, Charlie Lichtenbaum

TL;DR

The paper tackles optimizing AI for the stochastic game 2048 using evolutionary training without fine-tuning or RL. It contrasts a two-agent metaprompting framework with a single-agent value-function refinement for MCTS, incorporating a rollback mechanism. The single-agent approach yields substantial gains (average +$473.2$ points per cycle, $\rho=0.607$) and visible strategic growth, whereas the two-agent framework shows limited improvement, illuminating limits of meta-prompting for self-improvement. The results highlight the potential of evolutionary refinement, particularly code-based function optimization with rollback, for robust decision-making in non-deterministic environments and suggest directions to stabilize learning further.

Abstract

Optimizing artificial intelligence (AI) for dynamic environments remains a fundamental challenge in machine learning research. In this paper, we examine evolutionary training methods for optimizing AI to solve the game 2048, a 2D sliding puzzle. 2048, with its mix of strategic gameplay and stochastic elements, presents an ideal playground for studying decision-making, long-term planning, and dynamic adaptation. We implemented two distinct systems: a two-agent metaprompting system where a "thinker" large language model (LLM) agent refines gameplay strategies for an "executor" LLM agent, and a single-agent system based on refining a value function for a limited Monte Carlo Tree Search. We also experimented with rollback features to avoid performance degradation. Our results demonstrate the potential of evolutionary refinement techniques in improving AI performance in non-deterministic environments. The single-agent system achieved substantial improvements, with an average increase of 473.2 points per cycle, and with clear upward trends (correlation $ρ$=0.607) across training cycles. The LLM's understanding of the game grew as well, shown in its development of increasingly advanced strategies. Conversely, the two-agent system did not garner much improvement, highlighting the inherent limits of meta-prompting.

Merge and Conquer: Evolutionarily Optimizing AI for 2048

TL;DR

The paper tackles optimizing AI for the stochastic game 2048 using evolutionary training without fine-tuning or RL. It contrasts a two-agent metaprompting framework with a single-agent value-function refinement for MCTS, incorporating a rollback mechanism. The single-agent approach yields substantial gains (average + points per cycle, ) and visible strategic growth, whereas the two-agent framework shows limited improvement, illuminating limits of meta-prompting for self-improvement. The results highlight the potential of evolutionary refinement, particularly code-based function optimization with rollback, for robust decision-making in non-deterministic environments and suggest directions to stabilize learning further.

Abstract

Optimizing artificial intelligence (AI) for dynamic environments remains a fundamental challenge in machine learning research. In this paper, we examine evolutionary training methods for optimizing AI to solve the game 2048, a 2D sliding puzzle. 2048, with its mix of strategic gameplay and stochastic elements, presents an ideal playground for studying decision-making, long-term planning, and dynamic adaptation. We implemented two distinct systems: a two-agent metaprompting system where a "thinker" large language model (LLM) agent refines gameplay strategies for an "executor" LLM agent, and a single-agent system based on refining a value function for a limited Monte Carlo Tree Search. We also experimented with rollback features to avoid performance degradation. Our results demonstrate the potential of evolutionary refinement techniques in improving AI performance in non-deterministic environments. The single-agent system achieved substantial improvements, with an average increase of 473.2 points per cycle, and with clear upward trends (correlation =0.607) across training cycles. The LLM's understanding of the game grew as well, shown in its development of increasingly advanced strategies. Conversely, the two-agent system did not garner much improvement, highlighting the inherent limits of meta-prompting.
Paper Structure (15 sections, 5 figures)

This paper contains 15 sections, 5 figures.

Table of Contents

  1. Introduction
  2. Related Works
  3. Monte Carlo Tree Search
  4. Iterative Training
  5. AI Optimization for 2048
  6. Methodology
  7. Baseline
  8. Metaprompting
  9. MCTS Refinement
  10. Results
  11. Two-Agent System
  12. One-Value Function
  13. Changing the Value Function
  14. Discussion
  15. Conclusion

Figures (5)

  • Figure 1: Program Flow of the MCTS Refinement System
  • Figure 2: Metaprompting system, Average Score vs. Cycle Number; average score per round with 95% confidence intervals over cycle progression. The red dashed line represents the trend (slope: 12.3 points/round)
  • Figure 3: Altered Value Function with Rollback, Average Score vs. Cycle Number; average score across training cycles with 95% confidence intervals. The red dashed line highlights a positive linear trend ($\rho$ = 0.607), indicating performance improvement as training progresses.
  • Figure 4: Distribution of highest tiles.
  • Figure 5: Score range distribution.