AlphaViT: A flexible game-playing AI for multiple games and variable board sizes

TL;DR

This work replaces AlphaZero's fixed-input CNN with Vision Transformers to create AlphaViT, AlphaViD, and AlphaVDA, enabling a single DNN with shared weights to play multiple board games and adapt to varying board sizes. The authors demonstrate that these ViT-based agents can be trained on one or more games and outperform traditional baselines while approaching AlphaZero's strength, with AlphaViT often closest to AlphaZero. They show meaningful transfer and fine-tuning benefits from small-board pretraining and that multitask training can match or exceed single-task performance. The results highlight the potential of transformer-based architectures for flexible, robust game-playing AI across dynamic environments and game variants.

Abstract

We present three game-playing agents incorporating Vision Transformers (ViT) into the AlphaZero framework: AlphaViT, AlphaViD (AlphaViT with a transformer decoder), and AlphaVDA (AlphaViD with learnable action embeddings). These agents can play multiple board games of varying sizes using a single neural network with shared weights, thus overcoming AlphaZero's limitation of fixed board sizes. AlphaViT employs only a transformer encoder, whereas AlphaViD and AlphaVDA incorporate both a transformer encoder and a decoder. In AlphaViD, the decoder processes outputs from the encoder, whereas AlphaVDA uses learnable embeddings as the decoder inputs. The additional decoder in AlphaViD and AlphaVDA provides flexibility to adapt to various action spaces and board sizes. Experimental results show that the proposed agents, trained on either individual games or on multiple games simultaneously, consistently outperform traditional algorithms, such as Minimax and Monte Carlo Tree Search. They approach the performance of AlphaZero despite relying on a single deep neural network (DNN) with shared weights. In particular, AlphaViT performs strongly across all evaluated games. Furthermore, fine-tuning the DNN with weights pre-trained on small board games accelerates convergence and improves performance, particularly in Gomoku. Interestingly, simultaneous training on multiple games yields performance comparable to, or even surpassing, that of single-game training. These results indicate the potential of transformer-based architectures for developing more flexible and robust game-playing AI agents that excel in multiple games and dynamic environments.

Figures (11)

• Figure 1: Overview of the decision process. The agents take a stack of board planes and the current player as input and determine the next move using Monte Carlo Tree Search (MCTS). MCTS explores the game tree using the value and policy probabilities provided by the DNN.
• Figure 2: AlphaViT architecture. The input board is divided into patches by a convolutional layer, projected to embeddings, and combined with value, game, and pass tokens. After adding position embeddings, the sequence is processed by a transformer encoder. Outputs are used by MLP heads to predict the state value and move probabilities.
• Figure 3: Architecture diagrams of AlphaViD (top) and AlphaVDA (bottom). The input board is divided into patches, embedded, and combined with value and game tokens, then processed by a transformer encoder. For policy prediction, encoder outputs are sent to a transformer decoder via interpolation. In AlphaViD, decoder inputs are generated from board-derived embeddings via a fully connected layer; in AlphaVDA, learnable action embeddings are used. MLP heads output the final value and move probabilities.
• Figure 4: Elo rating progression over training iterations for AlphaViT, AlphaViD, and AlphaVDA in large board configurations (Connect 4, Gomoku, and Othello). The left column represents single-game-trained agents (single-task agents), whereas the right column indicates multi-game-trained agents (multitask agents). Solid lines represent Elo ratings calculated directly from aggregated game outcomes without employing bootstrapping, while shaded bands correspond to the 95% confidence intervals around these Elo ratings.
• Figure 5: Elo rating progression over training iterations for AlphaViT, AlphaViD, and AlphaVDA in their small board configurations (Connect 4 5x4, Gomoku 6x6, and Othello 6x6). Solid lines represent Elo ratings calculated directly from aggregated game outcomes without employing bootstrapping, while shaded bands correspond to the 95% confidence intervals around these Elo ratings.