Improving LLM Interpretability and Performance via Guided Embedding Refinement for Sequential Recommendation
Nanshan Jia, Chenfei Yuan, Yuhang Wu, Zeyu Zheng
TL;DR
This work tackles the interpretability and performance challenges of incorporating LLMs into sequential recommendation. It introduces guided embedding refinement, where a pre-trained LLM is fine-tuned to produce domain-specific attribute scores for items and users, which are normalized and fused with reduced-dimension base embeddings to form refined representations $e_i^r$ and $e_u^r$ that drive the recommender. The LLM fine-tuning combines a recommendation loss and a format loss via LoRA, while item guided embeddings are frozen and user guided embeddings are trainable. Empirically, refined embeddings consistently outperform base embeddings across diverse base models (GRU4Rec, SASRec, BERT4Rec) and domains (Movies, Clothing, Games), with improvements up to roughly 50% in MRR, Recall, and NDCG, and provide tangible interpretability via attribute-driven case studies. The results demonstrate a flexible, generalizable approach to integrate language models into sequential recommendation without sacrificing speed, offering a practical path toward more transparent and effective systems.
Abstract
The fast development of Large Language Models (LLMs) offers growing opportunities to further improve sequential recommendation systems. Yet for some practitioners, integrating LLMs to their existing base recommendation systems raises questions about model interpretability, transparency and related safety. To partly alleviate challenges from these questions, we propose guided embedding refinement, a method that carries out a guided and interpretable usage of LLM to enhance the embeddings associated with the base recommendation system. Instead of directly using LLMs as the backbone of sequential recommendation systems, we utilize them as auxiliary tools to emulate the sales logic of recommendation and generate guided embeddings that capture domain-relevant semantic information on interpretable attributes. Benefiting from the strong generalization capabilities of the guided embedding, we construct refined embedding by using the guided embedding and reduced-dimension version of the base embedding. We then integrate the refined embedding into the recommendation module for training and inference. A range of numerical experiments demonstrate that guided embedding is adaptable to various given existing base embedding models, and generalizes well across different recommendation tasks. The numerical results show that the refined embedding not only improves recommendation performance, achieving approximately $10\%$ to $50\%$ gains in Mean Reciprocal Rank (MRR), Recall rate, and Normalized Discounted Cumulative Gain (NDCG), but also enhances interpretability, as evidenced by case studies.