Predicting Human Chess Moves: An AI Assisted Analysis of Chess Games Using Skill-group Specific n-gram Language Models

TL;DR

The paper tackles the variability in human chess moves by reframing move prediction as a behavioral analysis task and training seven skill-group-specific 5-gram models on Lichess data. A model selector assigns games to the most appropriate skill-level model, and a Move Predictor generates next moves using that model, enabling real-time, human-centric prediction. Key findings show the selector achieves up to 31.7% accuracy on early-game data and Top-3 predictions outperform a benchmark by up to 39.1%, albeit with modest Top-1 gains and notable middle-game uncertainty. The approach is computationally efficient and practical for live chess analysis, though it is limited by short history windows and lack of game-state awareness, suggesting avenues for future sequential and stylistic open- Classifications.

Abstract

Chess, a deterministic game with perfect information, has long served as a benchmark for studying strategic decision-making and artificial intelligence. Traditional chess engines or tools for analysis primarily focus on calculating optimal moves, often neglecting the variability inherent in human chess playing, particularly across different skill levels. To overcome this limitation, we propose a novel and computationally efficient move prediction framework that approaches chess move prediction as a behavioral analysis task. The framework employs n-gram language models to capture move patterns characteristic of specific player skill levels. By dividing players into seven distinct skill groups, from novice to expert, we trained separate models using data from the open-source chess platform Lichess. The framework dynamically selects the most suitable model for prediction tasks and generates player moves based on preceding sequences. Evaluation on real-world game data demonstrates that the model selector module within the framework can classify skill levels with an accuracy of up to 31.7\% when utilizing early game information (16 half-moves). The move prediction framework also shows substantial accuracy improvements, with our Selector Assisted Accuracy being up to 39.1\% more accurate than our benchmark accuracy. The computational efficiency of the framework further enhances its suitability for real-time chess analysis.

Figures (6)

• Figure 1: The distribution considered for analysis
• Figure 2: Overview of the Chess Move Prediction Framework: The move prediction process relies on the model selected by the model selector, ensuring that the chosen model accurately reflects the player's skill level.
• Figure 3: Perplexity for each of the seven language models queried on each of the seven test sets
• Figure 4: Average by move surprisal for models over L1 games for 100 half moves
• Figure 5: TOP-1 Move Prediction Performance Comparison. The plot shows the accuracy for different half moves, comparing the Selector Selected Accuracy and the Benchmark Accuracy.