Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Predicting User Perception of Move Brilliance in Chess

Kamron Zaidi, Michael Guerzhoy

TL;DR

The paper tackles predicting human-perceived brilliance in chess moves by fusing engine outputs with rich game-tree shape features. It introduces an AggReduce-based neural network and a comprehensive feature pipeline to classify moves as brilliant, achieving 79% cross-validated accuracy and 83% PPV, 75% NPV, using data collected from Lichess annotations. A key finding is that brilliant moves are often non-obvious, as weaker engines may undervalue them, highlighting a distinction between strength and aesthetics. This work enables computer chess to display human-like brilliance and offers a data-driven framework for aesthetic assessment in tree-search domains.

Abstract

AI research in chess has been primarily focused on producing stronger agents that can maximize the probability of winning. However, there is another aspect to chess that has largely gone unexamined: its aesthetic appeal. Specifically, there exists a category of chess moves called ``brilliant" moves. These moves are appreciated and admired by players for their high intellectual aesthetics. We demonstrate the first system for classifying chess moves as brilliant. The system uses a neural network, using the output of a chess engine as well as features that describe the shape of the game tree. The system achieves an accuracy of 79% (with 50% base-rate), a PPV of 83%, and an NPV of 75%. We demonstrate that what humans perceive as ``brilliant" moves is not merely the best possible move. We show that a move is more likely to be predicted as brilliant, all things being equal, if a weaker engine considers it lower-quality (for the same rating by a stronger engine). Our system opens the avenues for computer chess engines to (appear to) display human-like brilliance, and, hence, creativity.

Predicting User Perception of Move Brilliance in Chess

TL;DR

The paper tackles predicting human-perceived brilliance in chess moves by fusing engine outputs with rich game-tree shape features. It introduces an AggReduce-based neural network and a comprehensive feature pipeline to classify moves as brilliant, achieving 79% cross-validated accuracy and 83% PPV, 75% NPV, using data collected from Lichess annotations. A key finding is that brilliant moves are often non-obvious, as weaker engines may undervalue them, highlighting a distinction between strength and aesthetics. This work enables computer chess to display human-like brilliance and offers a data-driven framework for aesthetic assessment in tree-search domains.

Abstract

AI research in chess has been primarily focused on producing stronger agents that can maximize the probability of winning. However, there is another aspect to chess that has largely gone unexamined: its aesthetic appeal. Specifically, there exists a category of chess moves called ``brilliant" moves. These moves are appreciated and admired by players for their high intellectual aesthetics. We demonstrate the first system for classifying chess moves as brilliant. The system uses a neural network, using the output of a chess engine as well as features that describe the shape of the game tree. The system achieves an accuracy of 79% (with 50% base-rate), a PPV of 83%, and an NPV of 75%. We demonstrate that what humans perceive as ``brilliant" moves is not merely the best possible move. We show that a move is more likely to be predicted as brilliant, all things being equal, if a weaker engine considers it lower-quality (for the same rating by a stronger engine). Our system opens the avenues for computer chess engines to (appear to) display human-like brilliance, and, hence, creativity.
Paper Structure (14 sections, 2 figures, 1 table)

This paper contains 14 sections, 2 figures, 1 table.

Table of Contents

  1. Introduction
  2. Literature Review
  3. Artificial Intelligence in Chess
  4. Chess Aesthetics
  5. Methods
  6. Data Collection
  7. Feature Engineering
  8. Classifier Models
  9. Results and Discussion
  10. Data Analysis
  11. Model Evaluation
  12. Brilliant moves more often elude the weaker engine
  13. Conclusions and Future Work
  14. Author Contributions

Figures (2)

  • Figure 1: AggReduce architecture. Features for each tree generated with a specific size and weight are processed individually in the first hidden layer, then they are aggregated in tiers.
  • Figure 2: Win chance using self-play weights (top) and Maia weights (bottom) for each move type vs number of nodes searched.