Budgeted Embedding Table For Recommender Systems

TL;DR

BET addresses the memory burden of fixed-size embedding tables in recommender systems by introducing table-level embedding size actions that strictly satisfy a budget constraint $1 - \| \mathbf{M}\|_{1,1} / ( (|\mathcal{U}|+|\mathcal{V}|) \cdot d_{max}) \le c$. It couples a bounded, probabilistic action sampler with a DeepSets-based fitness predictor to efficiently search for high-quality embedding configurations without per-user/per-item retraining. The approach is model-agnostic and demonstrated to achieve state-of-the-art performance under memory budgets on Gowalla and Yelp2018 across three backbone models (NCF, NGCF, LightGCN). Contributions include (i) a principled table-level action mechanism, (ii) a set-based action representation and inductive fitness predictor, (iii) diversified action selection and selective retraining, and (iv) extensive empirical validation showing superior accuracy at fixed memory budgets. This advances practical deployment of lightweight embeddings by balancing quality and memory with a scalable, learnable search process.

Abstract

At the heart of contemporary recommender systems (RSs) are latent factor models that provide quality recommendation experience to users. These models use embedding vectors, which are typically of a uniform and fixed size, to represent users and items. As the number of users and items continues to grow, this design becomes inefficient and hard to scale. Recent lightweight embedding methods have enabled different users and items to have diverse embedding sizes, but are commonly subject to two major drawbacks. Firstly, they limit the embedding size search to optimizing a heuristic balancing the recommendation quality and the memory complexity, where the trade-off coefficient needs to be manually tuned for every memory budget requested. The implicitly enforced memory complexity term can even fail to cap the parameter usage, making the resultant embedding table fail to meet the memory budget strictly. Secondly, most solutions, especially reinforcement learning based ones derive and optimize the embedding size for each each user/item on an instance-by-instance basis, which impedes the search efficiency. In this paper, we propose Budgeted Embedding Table (BET), a novel method that generates table-level actions (i.e., embedding sizes for all users and items) that is guaranteed to meet pre-specified memory budgets. Furthermore, by leveraging a set-based action formulation and engaging set representation learning, we present an innovative action search strategy powered by an action fitness predictor that efficiently evaluates each table-level action. Experiments have shown state-of-the-art performance on two real-world datasets when BET is paired with three popular recommender models under different memory budgets.

Figures (6)

• Figure 1: Overview of BET.
• Figure 2: Example of our set-based action formulation. Suppose we have three legitimate embedding sizes $\{16, 32, 64\}$, four users (red) and two items (green). Then, size 16 is a set containing user 1, user 2 and item 4, which means their embedding sizes are 16. Similarly, the action specifies the embedding sizes of user 3 and item 5 to be 32. Size 64 is not used by any users/items.
• Figure 3: Overview of the DeepSets-based fitness predictor.
• Figure 4: Sensitivity analysis of $m$ on Recall@20 and NDCG@20 with LightGCN as the backbone recommender.
• Figure 5: Sensitivity analysis of $T$ on Recall@20 and NDCG@20 with LightGCN as the backbone recommender.