Compositions of Variant Experts for Integrating Short-Term and Long-Term Preferences

TL;DR

This work tackles the challenge of balancing short-term session signals with long-term user preferences in sequential recommendation. It introduces Compositions of Variant Experts (CoVE), a mixture-of-experts framework with two aggregation schemes (CoVE$_h$ at the hidden representation level and CoVE$_s$ at the scoring level) that dynamically gates diverse short- and long-term models. Empirical results on Diginetica, RetailRocket, and Cosmetics show CoVE variants consistently outperform strong baselines (including GRU4Rec, SASRec, BPR, and LightGCN) on top-k ranking metrics, with LLM-based gating (OpenP5, ChatGPT-4o) offering limited gains in pure ID-based settings. The findings underscore the value of adaptive, context-dependent weighting of heterogeneous experts and suggest practical benefits when leveraging pretrained experts for scalability and efficiency.

Abstract

In the online digital realm, recommendation systems are ubiquitous and play a crucial role in enhancing user experience. These systems leverage user preferences to provide personalized recommendations, thereby helping users navigate through the paradox of choice. This work focuses on personalized sequential recommendation, where the system considers not only a user's immediate, evolving session context, but also their cumulative historical behavior to provide highly relevant and timely recommendations. Through an empirical study conducted on diverse real-world datasets, we have observed and quantified the existence and impact of both short-term (immediate and transient) and long-term (enduring and stable) preferences on users' historical interactions. Building on these insights, we propose a framework that combines short- and long-term preferences to enhance recommendation performance, namely Compositions of Variant Experts (CoVE). This novel framework dynamically integrates short- and long-term preferences through the use of different specialized recommendation models (i.e., experts). Extensive experiments showcase the effectiveness of the proposed methods and ablation studies further investigate the impact of variant expert types.

Figures (6)

• Figure 1: User A might want to be recommended a rock song based on their long-term interest in rock and hiphop, rather than a pop song based on their short-term interest in a trending pop song.
• Figure 2: An illustrative example distinguishing inputs for long-term and short-term preference models.
• Figure 3: Histogram of user preference values
• Figure 4: Overall flow of CoVE$_h$ variant. All experts receive user identity $u$ with their historical interactions $C_{1:T}^u$ and candidate item $p$. Each expert $i$ then returns the current context representation $h_i(u,T)$, item embeddings $\Psi_i(p)$, and item biases $\beta_i(p)$.
• Figure 5: Overall flow of CoVE$_s$ variant. All experts receive user identity $u$ with their historical interactions $C_{1:T}^u$ and candidate item $p$. Here, each expert $i$ just returns its predicted scores for the set of candidate items $\hat{\mathbf{r}}_i$.

Theorems & Definitions (2)

• definition 1
• definition 2