Optimizing Hearthstone Agents using an Evolutionary Algorithm

TL;DR

The use of evolutionary algorithms to develop agents who play a card game, Hearthstone, by optimizing a data-driven decision-making mechanism that takes into account all the elements currently in play shows how evolutionary computation could represent a considerable advantage in developing AIs for collectible card games such as Hearthstone.

Abstract

Digital collectible card games are not only a growing part of the video game industry, but also an interesting research area for the field of computational intelligence. This game genre allows researchers to deal with hidden information, uncertainty and planning, among other aspects. This paper proposes the use of evolutionary algorithms (EAs) to develop agents who play a card game, Hearthstone, by optimizing a data-driven decision-making mechanism that takes into account all the elements currently in play. Agents feature self-learning by means of a competitive coevolutionary training approach, whereby no external sparring element defined by the user is required for the optimization process. One of the agents developed through the proposed approach was runner-up (best 6%) in an international Hearthstone Artificial Intelligence (AI) competition. Our proposal performed remarkably well, even when it faced state-of-the-art techniques that attempted to take into account future game states, such as Monte-Carlo Tree search. This outcome shows how evolutionary computation could represent a considerable advantage in developing AIs for collectible card games such as Hearthstone.

Figures (11)

• Figure 1: Examples of Hearthstone cards
• Figure 2: Boxplots of the fitness distribution of the individuals in each generation (from all 10 ($E$) runs).
• Figure 3: Boxplots describing individuals' lifespan over all the 10 ($E$) runs. (AGE) describes the age of all individuals produced and (FIRST) shows the first generation that an individual entered the main population, thus being able to defeat at the very least the weakest individual in the previous population
• Figure 4: Boxplots of the age distribution of all individuals in each generation (from all 10 ($E$) runs).
• Figure 5: Average number of new individuals that enter in the population in each generation (from all 10 ($E$) runs). The lightgrey lines show 95% confidence intervals.