Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Models

Hailin Zhang, Zirui Liu, Boxuan Chen, Yikai Zhao, Tong Zhao, Tong Yang, Bin Cui

TL;DR

CAFE, a Compact, Adaptive, and Fast Embedding compression framework that addresses the growing memory demands of embedding tables in Deep Learning Recommendation Models, and proposes a fast and lightweight sketch data structure, named HotSketch, to capture feature importance and report hot features in real time.

Abstract

Recently, the growing memory demands of embedding tables in Deep Learning Recommendation Models (DLRMs) pose great challenges for model training and deployment. Existing embedding compression solutions cannot simultaneously meet three key design requirements: memory efficiency, low latency, and adaptability to dynamic data distribution. This paper presents CAFE, a Compact, Adaptive, and Fast Embedding compression framework that addresses the above requirements. The design philosophy of CAFE is to dynamically allocate more memory resources to important features (called hot features), and allocate less memory to unimportant ones. In CAFE, we propose a fast and lightweight sketch data structure, named HotSketch, to capture feature importance and report hot features in real time. For each reported hot feature, we assign it a unique embedding. For the non-hot features, we allow multiple features to share one embedding by using hash embedding technique. Guided by our design philosophy, we further propose a multi-level hash embedding framework to optimize the embedding tables of non-hot features. We theoretically analyze the accuracy of HotSketch, and analyze the model convergence against deviation. Extensive experiments show that CAFE significantly outperforms existing embedding compression methods, yielding 3.92% and 3.68% superior testing AUC on Criteo Kaggle dataset and CriteoTB dataset at a compression ratio of 10000x. The source codes of CAFE are available at GitHub.

CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Models

TL;DR

CAFE, a Compact, Adaptive, and Fast Embedding compression framework that addresses the growing memory demands of embedding tables in Deep Learning Recommendation Models, and proposes a fast and lightweight sketch data structure, named HotSketch, to capture feature importance and report hot features in real time.

Abstract

Recently, the growing memory demands of embedding tables in Deep Learning Recommendation Models (DLRMs) pose great challenges for model training and deployment. Existing embedding compression solutions cannot simultaneously meet three key design requirements: memory efficiency, low latency, and adaptability to dynamic data distribution. This paper presents CAFE, a Compact, Adaptive, and Fast Embedding compression framework that addresses the above requirements. The design philosophy of CAFE is to dynamically allocate more memory resources to important features (called hot features), and allocate less memory to unimportant ones. In CAFE, we propose a fast and lightweight sketch data structure, named HotSketch, to capture feature importance and report hot features in real time. For each reported hot feature, we assign it a unique embedding. For the non-hot features, we allow multiple features to share one embedding by using hash embedding technique. Guided by our design philosophy, we further propose a multi-level hash embedding framework to optimize the embedding tables of non-hot features. We theoretically analyze the accuracy of HotSketch, and analyze the model convergence against deviation. Extensive experiments show that CAFE significantly outperforms existing embedding compression methods, yielding 3.92% and 3.68% superior testing AUC on Criteo Kaggle dataset and CriteoTB dataset at a compression ratio of 10000x. The source codes of CAFE are available at GitHub.
Paper Structure (38 sections, 6 theorems, 4 equations, 18 figures, 2 tables)

This paper contains 38 sections, 6 theorems, 4 equations, 18 figures, 2 tables.

Table of Contents

  1. introduction
  2. Background and Motivation
  3. Limitations of Prior Art
  4. Our Proposed Method
  5. Main Contribution
  6. Preliminary
  7. DLRM
  8. Embedding Compression
  9. CAFE Design
  10. CAFE Overview
  11. The HotSketch Algorithm
  12. Migration Strategy
  13. Multi-level Hash Embedding
  14. Theoretical Analysis
  15. Accuracy of HotSketch
  16. ...and 23 more sections

Key Result

theorem 1

Given a data stream with $n$ features, and suppose their importance score vector is $a=\{a_1, a_2,\cdots,a_n\}$, where $a_1\geqslant a_2\geqslant\cdots\geqslant a_n$. Suppose that our HotSketch has $w$ buckets, and each bucket contains $c$ cells. Without distribution assumption, for a hot feature wi

Figures (18)

  • Figure 1: Overview of DLRM.
  • Figure 2: KL divergence between distributions on each day.
  • Figure 3: Comparing gradient norm and Zipf distributions.
  • Figure 4: Overview of CAFE.
  • Figure 5: The HotSketch algorithm.
  • ...and 13 more figures

Theorems & Definitions (6)

  • theorem 1
  • lemma 1
  • theorem 2
  • corollary 1
  • corollary 2
  • theorem 3