The Future is Sparse: Embedding Compression for Scalable Retrieval in Recommender Systems
Petr Kasalický, Martin Spišák, Vojtěch Vančura, Daniel Bohuněk, Rodrigo Alves, Pavel Kordík
TL;DR
This paper tackles the memory and throughput bottlenecks of industry-scale recommender systems caused by dense, high-dimensional embeddings. It proposes CompresSAE, a sparse autoencoder that projects dense $d$-dimensional embeddings into a high-dimensional ($h$) space with $k$ nonzeros, trained with a cosine reconstruction objective and retrieval-focused design, without retraining the upstream encoder. Inference stores the sparse embeddings in CSR and enables $O(k)$ dot-product retrieval in the sparse space, with an additional reconstruction-based path that uses a kernel trick for higher-fidelity similarity estimates. Experiments on a catalog of $ ext{O}(10^8)$ items show superior compression–accuracy trade-offs versus Matryoshka, and online CTR evaluations demonstrate minimal loss relative to uncompressed models while outperforming comparable compressed baselines, highlighting sparsity as a practical path to scalable retrieval.
Abstract
Industry-scale recommender systems face a core challenge: representing entities with high cardinality, such as users or items, using dense embeddings that must be accessible during both training and inference. However, as embedding sizes grow, memory constraints make storage and access increasingly difficult. We describe a lightweight, learnable embedding compression technique that projects dense embeddings into a high-dimensional, sparsely activated space. Designed for retrieval tasks, our method reduces memory requirements while preserving retrieval performance, enabling scalable deployment under strict resource constraints. Our results demonstrate that leveraging sparsity is a promising approach for improving the efficiency of large-scale recommenders. We release our code at https://github.com/recombee/CompresSAE.