Oracle-guided Dynamic User Preference Modeling for Sequential Recommendation

TL;DR

Oracle4Rec tackles the challenge of dynamic user preference modeling by incorporating future, oracle-like information during training. It introduces a dual-encoder architecture (Past and Future Information Encoders) with a lightweight Noise Filtering module and a causal self-attention backbone to extract reliable temporal representations, plus an Oracle-Guiding Module that minimizes a discrepancy between past and future embeddings using a KL-based measure. A tailored 2PTraining strategy first learns the future information before guiding the past representation, yielding forward-looking models that better capture evolving preferences. Empirical results on six real-world datasets show Oracle4Rec outperforms state-of-the-art baselines and can be plugged into other sequential methods to provide substantial gains, highlighting its practical impact for sequential recommendation systems.

Abstract

Sequential recommendation methods can capture dynamic user preferences from user historical interactions to achieve better performance. However, most existing methods only use past information extracted from user historical interactions to train the models, leading to the deviations of user preference modeling. Besides past information, future information is also available during training, which contains the ``oracle'' user preferences in the future and will be beneficial to model dynamic user preferences. Therefore, we propose an oracle-guided dynamic user preference modeling method for sequential recommendation (Oracle4Rec), which leverages future information to guide model training on past information, aiming to learn ``forward-looking'' models. Specifically, Oracle4Rec first extracts past and future information through two separate encoders, then learns a forward-looking model through an oracle-guiding module which minimizes the discrepancy between past and future information. We also tailor a two-phase model training strategy to make the guiding more effective. Extensive experiments demonstrate that Oracle4Rec is superior to state-of-the-art sequential methods. Further experiments show that Oracle4Rec can be leveraged as a generic module in other sequential recommendation methods to improve their performance with a considerable margin.

Figures (6)

• Figure 1: The architecture of Oracle4Rec, which consists of three parts: Past Information Encoder, Future Information Encoder and Oracle-Guiding Module. The first two encoders are both composed of Embedding Look-up Layer, Noise Filtering Module, Causal Self-Attention Module and Interaction Prediction Layer. For the ease of presentation, the Oracle-Guiding Module is set to minimize the distance between past information and future information in a 3D coordinate system.
• Figure 2: Four preference distributions of user 260 (the upper figure) and 869 (the lower figure) on ML100K. "dist." means distribution, and "PIE" is past information encoder, i.e., Oracle4Rec w/o Future information Encoder.
• Figure 3: The losses of past information encoder when using past information only or past and future information together on ML100K (left) and ML1M (right) datasets.
• Figure 4: Sensitivity analysis of three important hyper-parameters in Oracle4Rec on Beauty.
• Figure 5: Additional cases of four preference distributions of randomly selected 6 users on ML100K. "dist." means distribution, and "PIE" is past information encoder, i.e., Oracle4Rec w/o Future information Encoder.