Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CADC: Encoding User-Item Interactions for Compressing Recommendation Model Training Data

Hossein Entezari Zarch, Abdulla Alshabanah, Chaoyi Jiang, Murali Annavaram

TL;DR

The paper tackles the challenge of training large deep learning recommender models on ever-larger datasets by preserving collaborative signals while significantly reducing data. It introduces Collaborative Aware Data Compression (CADC), which first uses Matrix Factorization to learn fixed user and item embeddings that summarize interaction histories, then trains a Two-Tower Neural Network (TTNN) on a small subset with these embeddings initialized and frozen. Empirical results on MovieLens 1M, MovieLens 10M, and Epinions show CADC achieves near full-dataset accuracy with only 10% of the data and substantial time savings, with the Init-Frz embedding strategy often delivering the best ranking metrics (HR@10, NDCG@10). The approach offers a scalable path to maintain predictive quality while dramatically reducing data and compute requirements in production DLRMs.

Abstract

Deep learning recommendation models (DLRMs) are at the heart of the current e-commerce industry. However, the amount of training data used to train these large models is growing exponentially, leading to substantial training hurdles. The training dataset contains two primary types of information: content-based information (features of users and items) and collaborative information (interactions between users and items). One approach to reduce the training dataset is to remove user-item interactions. But that significantly diminishes collaborative information, which is crucial for maintaining accuracy due to its inclusion of interaction histories. This loss profoundly impacts DLRM performance. This paper makes an important observation that if one can capture the user-item interaction history to enrich the user and item embeddings, then the interaction history can be compressed without losing model accuracy. Thus, this work, Collaborative Aware Data Compression (CADC), takes a two-step approach to training dataset compression. In the first step, we use matrix factorization of the user-item interaction matrix to create a novel embedding representation for both the users and items. Once the user and item embeddings are enriched by the interaction history information the approach then applies uniform random sampling of the training dataset to drastically reduce the training dataset size while minimizing model accuracy drop. The source code of CADC is available at \href{https://anonymous.4open.science/r/DSS-RM-8C1D/README.md}{https://anonymous.4open.science/r/DSS-RM-8C1D/README.md}.

CADC: Encoding User-Item Interactions for Compressing Recommendation Model Training Data

TL;DR

The paper tackles the challenge of training large deep learning recommender models on ever-larger datasets by preserving collaborative signals while significantly reducing data. It introduces Collaborative Aware Data Compression (CADC), which first uses Matrix Factorization to learn fixed user and item embeddings that summarize interaction histories, then trains a Two-Tower Neural Network (TTNN) on a small subset with these embeddings initialized and frozen. Empirical results on MovieLens 1M, MovieLens 10M, and Epinions show CADC achieves near full-dataset accuracy with only 10% of the data and substantial time savings, with the Init-Frz embedding strategy often delivering the best ranking metrics (HR@10, NDCG@10). The approach offers a scalable path to maintain predictive quality while dramatically reducing data and compute requirements in production DLRMs.

Abstract

Deep learning recommendation models (DLRMs) are at the heart of the current e-commerce industry. However, the amount of training data used to train these large models is growing exponentially, leading to substantial training hurdles. The training dataset contains two primary types of information: content-based information (features of users and items) and collaborative information (interactions between users and items). One approach to reduce the training dataset is to remove user-item interactions. But that significantly diminishes collaborative information, which is crucial for maintaining accuracy due to its inclusion of interaction histories. This loss profoundly impacts DLRM performance. This paper makes an important observation that if one can capture the user-item interaction history to enrich the user and item embeddings, then the interaction history can be compressed without losing model accuracy. Thus, this work, Collaborative Aware Data Compression (CADC), takes a two-step approach to training dataset compression. In the first step, we use matrix factorization of the user-item interaction matrix to create a novel embedding representation for both the users and items. Once the user and item embeddings are enriched by the interaction history information the approach then applies uniform random sampling of the training dataset to drastically reduce the training dataset size while minimizing model accuracy drop. The source code of CADC is available at \href{https://anonymous.4open.science/r/DSS-RM-8C1D/README.md}{https://anonymous.4open.science/r/DSS-RM-8C1D/README.md}.
Paper Structure (10 sections, 3 equations, 1 figure, 2 tables)

This paper contains 10 sections, 3 equations, 1 figure, 2 tables.

Table of Contents

  1. Introduction
  2. Collaborative Aware Data Compression
  3. Pre-training Embedding Vectors
  4. Integrating Enhanced Embedding Vectors into DLRM
  5. Experimental Setup
  6. Baselines
  7. Results
  8. Sensitivity Analysis
  9. Related Works
  10. Conclusion

Figures (1)

  • Figure 1: Impact of Data Filtering Ratios on HR@10 for CADC. This plot demonstrates how the performance of CADC is influenced by varying levels of data reduction.