CollectiveKV: Decoupling and Sharing Collaborative Information in Sequential Recommendation

TL;DR

CollectiveKV addresses latency and storage challenges in Transformer-based sequential recommendations by explicitly sharing KV representations across users. It decomposes KV into a user-specific, low-dimensional component and a cross-user, high-dimensional collective component stored in a global KV pool, with a learnable router selecting shared indices. This cross-user sharing achieves dramatic KV cache compression (down to about 0.8% of the original size in some settings) while preserving or even improving predictive performance and reducing latency across multiple models and datasets. The approach leverages collaborative signals inherent in user behavior, offering a scalable, low-latency solution for long-horizon sequential recommendation.

Abstract

Sequential recommendation models are widely used in applications, yet they face stringent latency requirements. Mainstream models leverage the Transformer attention mechanism to improve performance, but its computational complexity grows with the sequence length, leading to a latency challenge for long sequences. Consequently, KV cache technology has recently been explored in sequential recommendation systems to reduce inference latency. However, KV cache introduces substantial storage overhead in sequential recommendation systems, which often have a large user base with potentially very long user history sequences. In this work, we observe that KV sequences across different users exhibit significant similarities, indicating the existence of collaborative signals in KV. Furthermore, we analyze the KV using singular value decomposition (SVD) and find that the information in KV can be divided into two parts: the majority of the information is shareable across users, while a small portion is user-specific. Motivated by this, we propose CollectiveKV, a cross-user KV sharing mechanism. It captures the information shared across users through a learnable global KV pool. During inference, each user retrieves high-dimensional shared KV from the pool and concatenates them with low-dimensional user-specific KV to obtain the final KV. Experiments on five sequential recommendation models and three datasets show that our method can compress the KV cache to only 0.8% of its original size, while maintaining or even enhancing model performance.

Figures (9)

• Figure 1: Probability density (estimated via kernel density estimation) of cosine similarities of K/V between different users on two datasets and three models.
• Figure 2: Probability density (estimated via kernel density estimation) of cosine similarity of principal K/V and residual K/V across users.
• Figure 3: Overall framework of CollectiveKV. All the network modules, including the router network, the linear projection layer, and the global K/V pool, have the same parameters across different users. During prefilling, user-specific K/V and global K/V indices are calculated and cached.
• Figure 4: The influence of the user-specific feature dimension on the model performance.
• Figure 5: The influence of the global K/V pool size on the model performance.