Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Generalized User Representations for Transfer Learning

Ghazal Fazelnia, Sanket Gupta, Claire Keum, Mark Koh, Ian Anderson, Mounia Lalmas

TL;DR

A novel framework for user representation in large-scale recommender systems, aiming at effectively representing diverse user taste in a generalized manner, using a two-stage methodology combining representation learning and transfer learning.

Abstract

We present a novel framework for user representation in large-scale recommender systems, aiming at effectively representing diverse user taste in a generalized manner. Our approach employs a two-stage methodology combining representation learning and transfer learning. The representation learning model uses an autoencoder that compresses various user features into a representation space. In the second stage, downstream task-specific models leverage user representations via transfer learning instead of curating user features individually. We further augment this methodology on the representation's input features to increase flexibility and enable reaction to user events, including new user experiences, in Near-Real Time. Additionally, we propose a novel solution to manage deployment of this framework in production models, allowing downstream models to work independently. We validate the performance of our framework through rigorous offline and online experiments within a large-scale system, showcasing its remarkable efficacy across multiple evaluation tasks. Finally, we show how the proposed framework can significantly reduce infrastructure costs compared to alternative approaches.

Generalized User Representations for Transfer Learning

TL;DR

A novel framework for user representation in large-scale recommender systems, aiming at effectively representing diverse user taste in a generalized manner, using a two-stage methodology combining representation learning and transfer learning.

Abstract

We present a novel framework for user representation in large-scale recommender systems, aiming at effectively representing diverse user taste in a generalized manner. Our approach employs a two-stage methodology combining representation learning and transfer learning. The representation learning model uses an autoencoder that compresses various user features into a representation space. In the second stage, downstream task-specific models leverage user representations via transfer learning instead of curating user features individually. We further augment this methodology on the representation's input features to increase flexibility and enable reaction to user events, including new user experiences, in Near-Real Time. Additionally, we propose a novel solution to manage deployment of this framework in production models, allowing downstream models to work independently. We validate the performance of our framework through rigorous offline and online experiments within a large-scale system, showcasing its remarkable efficacy across multiple evaluation tasks. Finally, we show how the proposed framework can significantly reduce infrastructure costs compared to alternative approaches.
Paper Structure (40 sections, 2 equations, 8 figures, 4 tables, 1 algorithm)

This paper contains 40 sections, 2 equations, 8 figures, 4 tables, 1 algorithm.

Table of Contents

  1. Introduction
  2. Related Works
  3. Approach
  4. Notations
  5. Architecture
  6. Modality Encoders
  7. Input Features
  8. Autoencoder for Core User Representation
  9. Model Implementation
  10. Training Dataset
  11. Input features
  12. Model Details and Retrain Cadence
  13. Transfer Learning Methodology
  14. Use Cases
  15. Classification models
  16. ...and 25 more sections

Figures (8)

  • Figure 1: User Representation Model Architecture. Catalog interactions are embedded via pre-trained modality encoders to serve as input to an autoencoder model. The latent representation, $z_u$, serves as the User Representation for disparate downstream recommendation tasks.
  • Figure 2: Embedding outputs of music modality encoders are aggregated over multiple time horizons.
  • Figure 3: Music modality encoders as well as podcast modality encoders can feed into the user representation.
  • Figure 4: Downstream model tasks before (left) and after (right) incorporating transfer learning with a user representation.
  • Figure 5: Near-Real Time Inference of User Representations. Cold-start user activations and user interactions trigger inferences, allowing for smooth updates to reflect the latest information.
  • ...and 3 more figures