LLM-based Bi-level Multi-interest Learning Framework for Sequential Recommendation

TL;DR

This work addresses noisy, sparse implicit feedback in multi-interest sequential recommendation by introducing EIMF, a bi-level framework that separately learns implicit behavioral interests (IBIM) with a traditional SR backbone and explicit semantic interests (ESIM) via clustering and LLM-driven inference during training. By aligning modalities through semantic prediction and a modality alignment task, ESIM enriches the behavioral representations without injecting LLMs into the online serving path, enabling LLM-free, low-latency recommendations. The approach achieves consistent gains across multiple real-world datasets and backbone models, with offline ESIM precomputation dramatically reducing online cost while preserving semantic richness. The proposed framework demonstrates a practical pathway to leverage LLM capabilities for recommendation systems at industrial scales, highlighting both performance improvements and efficiency gains.

Abstract

Sequential recommendation (SR) leverages users' dynamic preferences, with recent advances incorporating multi-interest learning to model diverse user interests. However, most multi-interest SR models rely on noisy, sparse implicit feedback, limiting recommendation accuracy. Large language models (LLMs) offer robust reasoning on low-quality data but face high computational costs and latency challenges for SR integration. We propose a novel LLM-based multi-interest SR framework combining implicit behavioral and explicit semantic perspectives. It includes two modules: the Implicit Behavioral Interest Module (IBIM), which learns from user behavior using a traditional SR model, and the Explicit Semantic Interest Module (ESIM), which uses clustering and prompt-engineered LLMs to extract semantic multi-interest representations from informative samples. Semantic insights from ESIM enhance IBIM's behavioral representations via modality alignment and semantic prediction tasks. During inference, only IBIM is used, ensuring efficient, LLM-free recommendations. Experiments on four real-world datasets validate the framework's effectiveness and practicality.

Figures (9)

• Figure 1: Overall Framework of EIMF.
• Figure 2: An example of a case that prompts design and LLM inference.
• Figure 3: The results of different $K$ on the Ml-1M dataset
• Figure 4: The results of ablation experiments on three datasets.
• Figure 5: The results of the AP clustering algorithm with different $p$ hyperparameter settings on three datasets.