Deviation Ratings: A General, Clone-Invariant Rating Method

TL;DR

This work addresses rating strategies in N-player general-sum normal-form games by introducing Deviation Rating, a clone-invariant, mixture-invariant, and offset-invariant method based on coarse correlated equilibria. By selecting for unique deviation gains through a sequence of linear programs, the approach avoids equilibrium-selection pitfalls while guaranteeing existence and uniqueness of ratings. The method is demonstrated across illustrative cyclic/coordination games and language-model evaluation datasets, showing that deviation ratings neutralize redundancy and reveal niche strengths, while remaining scalable to large, real-world evaluation data. The practical impact is a robust, data-agnostic framework for evaluating multi-agent systems (notably LLMs) that can guide targeted model improvement without curation or vulnerability to clone attacks.

Abstract

Many real-world multi-agent or multi-task evaluation scenarios can be naturally modelled as normal-form games due to inherent strategic (adversarial, cooperative, and mixed motive) interactions. These strategic interactions may be agentic (e.g. players trying to win), fundamental (e.g. cost vs quality), or complementary (e.g. niche finding and specialization). In such a formulation, it is the strategies (actions, policies, agents, models, tasks, prompts, etc.) that are rated. However, the rating problem is complicated by redundancy and complexity of N-player strategic interactions. Repeated or similar strategies can distort ratings for those that counter or complement them. Previous work proposed ``clone invariant'' ratings to handle such redundancies, but this was limited to two-player zero-sum (i.e. strictly competitive) interactions. This work introduces the first N-player general-sum clone invariant rating, called deviation ratings, based on coarse correlated equilibria. The rating is explored on several domains including LLMs evaluation.

Figures (5)

• Figure 1: Population ratings for Shapley's game. The position of the points indicates the underlying mixture of each strategy. The fill color of the point represents its rating under the rating function. The outline color represents the marginal probability mass each strategy has under the equilibrium. Each column is a different population distribution. Top: Uniform. Bottom: CCE.
• Figure 2: Livebench analysis. (a) Model ratings with competing evaluation algorithms. Uniform and Elo ratings have been rescaled to fit into the same domain as the CCE deviation ratings. (b) Analysis showing how the four most salient tasks contributes to the CCE deviation rating. The bars sum to the corresponding ratings. (c) The full raw model vs task data used for evaluation.
• Figure 3: Model improvement analysis. Shows the equilibrium gap (left axis, lower better) and average payoffs (right axis, higher better) with iteration count over two OpenSpiel lanctot2019_openspiel environments.
• Figure 4: RL agents on Atari learning environments. The agents are rated in two gamification regimes: two-player (2P) zero-sum agent vs task, and three-player (3P) agent vs agent vs task. We evaluate using uniform and deviation ratings. The agents are ordered according to their uniform rating. We normalized all the ratings to be between $-1$ and $0$ (higher is better).
• Figure : Comparison of rating methods.