Online Learning of Counter Categories and Ratings in PvP Games

TL;DR

The paper addresses the limitation of scalar strength ratings in capturing intransitive relationships in PvP games by proposing Elo-RCC, an online extension of Elo that learns discrete counter categories via an EM-like procedure. It combines real-time Elo updates with a learnable $M\times M$ counter table and category distributions, updating both ratings and counter relationships after each match. The method yields comparable accuracy to Neural Counter Table (NCT) on benchmarks while enabling online updates and maintaining interpretability; smaller counter spaces ($M\in\{3,9\}$) can even surpass NCT in some settings. This approach offers a practical balance analysis tool for game balance and matchmaking and suggests avenues for future work in more complex team dynamics and non-zero-sum environments.

Abstract

In competitive games, strength ratings like Elo are widely used to quantify player skill and support matchmaking by accounting for skill disparities better than simple win rate statistics. However, scalar ratings cannot handle complex intransitive relationships, such as counter strategies seen in Rock-Paper-Scissors. To address this, recent work introduced Neural Rating Table and Neural Counter Table, which combine scalar ratings with discrete counter categories to model intransitivity. While effective, these methods rely on neural network training and cannot perform real-time updates. In this paper, we propose an online update algorithm that extends Elo principles to incorporate real-time learning of counter categories. Our method dynamically adjusts both ratings and counter relationships after each match, preserving the explainability of scalar ratings while addressing intransitivity. Experiments on zero-sum competitive games demonstrate its practicality, particularly in scenarios without complex team compositions.

Figures (4)

• Figure 1: An illustration of counter relationships in games. While Player 2's Elo rating may be slightly higher than Player 1's, the dominance relationship between strategies A and B remains unclear due to potential counter dynamics, emphasizing the limitations of scalar ratings in capturing such complexities.
• Figure 2: Neural Rating Table (NRT) predicts win probabilities between two teams using shared-weight encoders and the Bradley-Terry model, with exponential activation applied in the rating encoder for reparameterization.
• Figure 3: Neural Counter Table (NCT) models counter relationships using category encoders and residual win value decoders.
• Figure 4: Illustration of the EM algorithm. Residual win values for all categories are learned iteratively, enabling the best-fitting category for each individual to be identified and refined as the classification label.