Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Efficient Feature Interactions with Transformers: Improving User Spending Propensity Predictions in Gaming

Ved Prakash, Kartavya Kothari

TL;DR

This paper benchmarks tree-based and deep-learning models that show good results on structured data, and proposes a new architecture change that surpasses the state-of-the-art FT-Transformer and surpasses the state-of-the-art FT-Transformer on the task of predicting the user's propensity to spend in a gaming round.

Abstract

Dream11 is a fantasy sports platform that allows users to create their own virtual teams for real-life sports events. We host multiple sports and matches for our 200M+ user base. In this RMG (real money gaming) setting, users pay an entry amount to participate in various contest products that we provide to users. In our current work, we discuss the problem of predicting the user's propensity to spend in a gaming round, so it can be utilized for various downstream applications. e.g. Upselling users by incentivizing them marginally as per their spending propensity, or personalizing the product listing based on the user's propensity to spend. We aim to model the spending propensity of each user based on past transaction data. In this paper, we benchmark tree-based and deep-learning models that show good results on structured data, and we propose a new architecture change that is specifically designed to capture the rich interactions among the input features. We show that our proposed architecture outperforms the existing models on the task of predicting the user's propensity to spend in a gaming round. Our new transformer model surpasses the state-of-the-art FT-Transformer, improving MAE by 2.5\% and MSE by 21.8\%.

Efficient Feature Interactions with Transformers: Improving User Spending Propensity Predictions in Gaming

TL;DR

This paper benchmarks tree-based and deep-learning models that show good results on structured data, and proposes a new architecture change that surpasses the state-of-the-art FT-Transformer and surpasses the state-of-the-art FT-Transformer on the task of predicting the user's propensity to spend in a gaming round.

Abstract

Dream11 is a fantasy sports platform that allows users to create their own virtual teams for real-life sports events. We host multiple sports and matches for our 200M+ user base. In this RMG (real money gaming) setting, users pay an entry amount to participate in various contest products that we provide to users. In our current work, we discuss the problem of predicting the user's propensity to spend in a gaming round, so it can be utilized for various downstream applications. e.g. Upselling users by incentivizing them marginally as per their spending propensity, or personalizing the product listing based on the user's propensity to spend. We aim to model the spending propensity of each user based on past transaction data. In this paper, we benchmark tree-based and deep-learning models that show good results on structured data, and we propose a new architecture change that is specifically designed to capture the rich interactions among the input features. We show that our proposed architecture outperforms the existing models on the task of predicting the user's propensity to spend in a gaming round. Our new transformer model surpasses the state-of-the-art FT-Transformer, improving MAE by 2.5\% and MSE by 21.8\%.
Paper Structure (18 sections, 10 equations, 3 figures, 3 tables)

This paper contains 18 sections, 10 equations, 3 figures, 3 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Problem-formulation
  4. Method
  5. Proximity-Aware Contextual Transformer
  6. Benchmark models
  7. Experiments and Results
  8. Scope of the comparison
  9. Implementation Details
  10. Comparison with Other models
  11. Performance Analysis
  12. How does MLP model perform ? Is it really a strong baseline ?
  13. Does performance improve with dataset size?
  14. Discussion
  15. Interpretation of Experimental Results
  16. ...and 3 more sections

Figures (3)

  • Figure 1: The Proximity-Aware Contextual Transformer architecture. Firstly, Feature Tokenizer transforms features to embeddings. The embeddings are then processed by the Transformer module and the final representation of the [CLS] token is used for prediction.
  • Figure 2: (a) Feature Tokenizer; in the example, there are three numerical and two categorical features; (b) One Transformer layer.
  • Figure 3: The graph compares the performance of the Proximity-Aware Contextual Transformer and the FT Transformer across different train sizes, represented as a fraction of n-observations. Error bars represent standard deviation from the mean across different runs, illustrating the variability due to the use of various random seeds.