Estimating Player Performance in Different Contexts Using Fine-tuned Large Events Models

TL;DR

The paper addresses how to quantify player impact across varying team contexts in soccer by introducing Large Event Models (LEMs) that simulate matches via event-level predictions conditioned on game state. It constructs a base LEM from broad event data and then fine-tunes it to reflect specific contexts (team, player, additions, replacements) using the WyScout 2017-2018 EPL dataset, enabling scenario analyses such as Ronaldo or Messi transfers. Key contributions include a structured fine-tuning framework, a transparent parameter-tuning protocol, and experiments demonstrating that context significantly shapes player impact and forecasted team performance. The work offers a data-driven tool for recruitment decision-making and scenario planning, highlighting both the potential and current limitations of using LEMs for context-aware soccer analytics.

Abstract

This paper introduces an innovative application of Large Event Models (LEMs), akin to Large Language Models, to the domain of soccer analytics. By learning the language of soccer - predicting variables for subsequent events rather than words - LEMs facilitate the simulation of matches and offer various applications, including player performance prediction across different team contexts. We focus on fine-tuning LEMs with the WyScout dataset for the 2017-2018 Premier League season to derive specific insights into player contributions and team strategies. Our methodology involves adapting these models to reflect the nuanced dynamics of soccer, enabling the evaluation of hypothetical transfers. Our findings confirm the effectiveness and limitations of LEMs in soccer analytics, highlighting the model's capability to forecast teams' expected standings and explore high-profile scenarios, such as the potential effects of transferring Cristiano Ronaldo or Lionel Messi to different teams in the Premier League. This analysis underscores the importance of context in evaluating player quality. While general metrics may suggest significant differences between players, contextual analyses reveal narrower gaps in performance within specific team frameworks.

Figures (7)

• Figure 1: LEMs and LLMs work on the same principle: given the initial context that contains all current information, it can forecast the next token. This token then updates the context iteratively until an exit criterion is met. For LLMs, the tokens are words. For LEMs, the tokens are events.
• Figure 2: This figure depicts the two-stage process of developing a fine-tuned LEM model: first, we use a large dataset to build a LEM, then we fine-tune it using specific data about our target.
• Figure 3: This figure illustrates that the outcome variability originates from the model's training and fine-tuning rather than the simulation process. At the threshold of 3000 simulations, a sequence of 10 consecutive wins is required to alter the expected points by a mere +0.01. Additionally, the figure shows that the error distribution follows a normal curve, indicating that, over an extended period, the average error in simulations is expected to converge towards zero.
• Figure 4: The expected impact of adding Cristiano Ronaldo or Lionel Messi on the teams in the EPL. The figure presents the violin plots of the simulations using the fine-tuned models.
• Figure 5: The expected impact of replacing a player for Cristiano Ronaldo or Lionel Messi on the teams in the EPL.