Development of an End-to-end Machine Learning System with Application to In-app Purchases

TL;DR

The paper presents an end-to-end ML system for predicting the timing of in-app purchases to personalize offers in King’s mobile games. It combines feature engineering, two primary models (a Tabular NN and a ContentRNN) and rigorous offline and online evaluation to demonstrate predictive gains, including a 20% uplift in a key business metric in an online test. The authors detail a robust production platform (KingML) spanning data ingestion, feature engineering, training, deployment, monitoring, and infrastructure (on Vertex AI and GCP services), along with retraining policies and deployment strategies. The work emphasizes practical impact, scalability, and ongoing improvements (feature stores, real-time predictions, and XAI) to support production ML at scale in gaming.

Abstract

Machine learning (ML) systems have become vital in the mobile gaming industry. Companies like King have been using them in production to optimize various parts of the gaming experience. One important area is in-app purchases: purchases made in the game by players in order to enhance and customize their gameplay experience. In this work we describe how we developed an ML system in order to predict when a player is expected to make their next in-app purchase. These predictions are used to present offers to players. We briefly describe the problem definition, modeling approach and results and then, in considerable detail, outline the end-to-end ML system. We conclude with a reflection on challenges encountered and plans for future work.

Figures (5)

• Figure 1: Discounted offer
• Figure 2: ContentRNN Architecture
• Figure 3: Offline results of both models on a proprietary test dataset.
• Figure 4: Structure of a (re) training pipeline. Purple colored elements represent standard operations in Vertex AI.
• Figure 5: Inference pipeline