Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

MALLOC: Benchmarking the Memory-aware Long Sequence Compression for Large Sequential Recommendation

Qihang Yu, Kairui Fu, Zhaocheng Du, Yuxuan Si, Kaiyuan Li, Weihao Zhao, Zhicheng Zhang, Jieming Zhu, Quanyu Dai, Zhenhua Dong, Shengyu Zhang, Kun Kuang, Fei Wu

TL;DR

MALLOC presents a comprehensive investigation and systematic classification of memory management techniques applicable to large sequential recommendations, integrated into state-of-the-art recommenders, enabling a reproducible and accessible evaluation platform.

Abstract

The scaling law, which indicates that model performance improves with increasing dataset and model capacity, has fueled a growing trend in expanding recommendation models in both industry and academia. However, the advent of large-scale recommenders also brings significantly higher computational costs, particularly under the long-sequence dependencies inherent in the user intent of recommendation systems. Current approaches often rely on pre-storing the intermediate states of the past behavior for each user, thereby reducing the quadratic re-computation cost for the following requests. Despite their effectiveness, these methods often treat memory merely as a medium for acceleration, without adequately considering the space overhead it introduces. This presents a critical challenge in real-world recommendation systems with billions of users, each of whom might initiate thousands of interactions and require massive memory for state storage. Fortunately, there have been several memory management strategies examined for compression in LLM, while most have not been evaluated on the recommendation task. To mitigate this gap, we introduce MALLOC, a comprehensive benchmark for memory-aware long sequence compression. MALLOC presents a comprehensive investigation and systematic classification of memory management techniques applicable to large sequential recommendations. These techniques are integrated into state-of-the-art recommenders, enabling a reproducible and accessible evaluation platform. Through extensive experiments across accuracy, efficiency, and complexity, we demonstrate the holistic reliability of MALLOC in advancing large-scale recommendation. Code is available at https://anonymous.4open.science/r/MALLOC.

MALLOC: Benchmarking the Memory-aware Long Sequence Compression for Large Sequential Recommendation

TL;DR

MALLOC presents a comprehensive investigation and systematic classification of memory management techniques applicable to large sequential recommendations, integrated into state-of-the-art recommenders, enabling a reproducible and accessible evaluation platform.

Abstract

The scaling law, which indicates that model performance improves with increasing dataset and model capacity, has fueled a growing trend in expanding recommendation models in both industry and academia. However, the advent of large-scale recommenders also brings significantly higher computational costs, particularly under the long-sequence dependencies inherent in the user intent of recommendation systems. Current approaches often rely on pre-storing the intermediate states of the past behavior for each user, thereby reducing the quadratic re-computation cost for the following requests. Despite their effectiveness, these methods often treat memory merely as a medium for acceleration, without adequately considering the space overhead it introduces. This presents a critical challenge in real-world recommendation systems with billions of users, each of whom might initiate thousands of interactions and require massive memory for state storage. Fortunately, there have been several memory management strategies examined for compression in LLM, while most have not been evaluated on the recommendation task. To mitigate this gap, we introduce MALLOC, a comprehensive benchmark for memory-aware long sequence compression. MALLOC presents a comprehensive investigation and systematic classification of memory management techniques applicable to large sequential recommendations. These techniques are integrated into state-of-the-art recommenders, enabling a reproducible and accessible evaluation platform. Through extensive experiments across accuracy, efficiency, and complexity, we demonstrate the holistic reliability of MALLOC in advancing large-scale recommendation. Code is available at https://anonymous.4open.science/r/MALLOC.
Paper Structure (24 sections, 2 equations, 5 figures, 4 tables)

This paper contains 24 sections, 2 equations, 5 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Large Recommender System
  4. Long‑Sequence Recommendation
  5. Benchmark Design
  6. Preliminary
  7. Baselines
  8. Sequence-Level Compression
  9. Token-Level Compression
  10. Head-Level Compression
  11. Precision-Level Compression
  12. Architecture Level
  13. Backbone
  14. Datasets
  15. Metrics
  16. ...and 9 more sections

Figures (5)

  • Figure 1: (a) Long-term historical interactions are crucial for capturing periodic user behaviors. (b) The standard self-attention mechanism in Large Recommender Systems.
  • Figure 2: Benchmark Overview.
  • Figure 3: A Visual Taxonomy of Multi-Level KV Cache Compression. (a) Transforms the full sequence of length $N$ into a compact representation of length $M$. (b) Reduces the token count by either merging adjacent tokens or pruning less significant ones. (c) Consolidates $H$ specific attention heads into $G$ shared or grouped KV heads to reduce memory redundancy. (d) Compresses the storage bit-width of KV states from high-precision to low-precision grids without altering the sequence length.
  • Figure 4: The Trade-off Landscape between Recommendation Performance and Resource Consumption. The visualization contrasts GAUC against computational cost (MACs) on KuaiVideo (Left) and Amazon (Right) datasets. The bubble size is proportional to the memory occupation.
  • Figure 5: Scalability Analysis on Amazon Dataset. The recommendation performance (GAUC, Logloss) of different memory-aware methods as the model depth increases from 1 to 32 blocks.