Policies of Multiple Skill Levels for Better Strength Estimation in Games

TL;DR

The paper addresses the challenge of accurately estimating human skill levels in games to tailor AI behavior. It enhances a prior strength estimator by incorporating move-policy priors across multiple skill levels and loss-based features, combining them in a stacked meta-model to predict ranks. Experiments on Go and chess show that the proposed method achieves higher accuracy than the previous state of the art (e.g., 80% at 10 matches and 92% at 20 matches in Go), with analogous gains in chess. The study also analyzes the role of priors across skill levels and the impact of losses, revealing practical benefits for both group- and player-specific rank estimation and highlighting implications for human–AI teaching and tutoring in games.

Abstract

Accurately estimating human skill levels is crucial for designing effective human-AI interactions so that AI can provide appropriate challenges or guidance. In games where AI players have beaten top human professionals, strength estimation plays a key role in adapting AI behavior to match human skill levels. In a previous state-of-the-art study, researchers have proposed a strength estimator trained using human players' match data. Given some matches, the strength estimator computes strength scores and uses them to estimate player ranks (skill levels). In this paper, we focus on the observation that human players' behavior tendency varies according to their strength and aim to improve the accuracy of strength estimation by taking this into account. Specifically, in addition to strength scores, we obtain policies for different skill levels from neural networks trained using human players' match data. We then combine features based on these policies with the strength scores to estimate strength. We conducted experiments on Go and chess. For Go, our method achieved an accuracy of 80% in strength estimation when given 10 matches, which increased to 92% when given 20 matches. In comparison, the previous state-of-the-art method had an accuracy of 71% with 10 matches and 84% with 20 matches, demonstrating improvements of 8-9%. We observed similar improvements in chess. These results contribute to developing a more accurate strength estimation method and to improving human-AI interaction.

Figures (15)

• Figure 1: An overview of our method, where $k$ is the number of moves played by a player in a match, $\beta_i$ denotes strength score of state-move pair $(s_i, m_i)$, $p_{j,i}$ denotes the selection probability of move $m_i$ at state $s_i$ from the imitation model at skill level $j$, and $l_i$ denotes the loss of move $m_i$, defined as the difference between the state evaluations before and after executing $m_i$: $l_i = v'_i - v_i$. Here, $v_i$ is the evaluation of state $s_i$, and $v'_i$ is the evaluation of the resulting state after playing move $m_i$, both computed by a strong AI player.
• Figure 2: The geometric means of priors, along with 95% confidence intervals, across various skill levels for different rank groups.
• Figure 3: The mean loss at each ply.
• Figure 4: The mean strength scores of each player in each rank group with $n=20$.
• Figure 5: The mean strength scores of randomly sampled data points in each rank group with $n=20$.