Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Break the ID-Language Barrier: An Adaption Framework for LLM-based Sequential Recommendation

Xiaohan Yu, Li Zhang, Xin Zhao, Yue Wang

TL;DR

This work addresses the challenge of integrating domain-specific ID knowledge into LLM-based sequential recommendation. It introduces IDLE-Adapter, a two-component framework combining hard prompts and a four-part adapter that maps sparse ID interactions into dense LLM-compatible representations, with layer-wise refinement and distribution alignment guided by MMD. The approach yields significant gains over state-of-the-art ID-based and LLM-based methods across multiple datasets, while showing robust generalization across LLM backbones and ID models and providing interpretable bridging behavior. The proposed framework offers a practical pathway to leverage rich domain knowledge inside LLMs for improved recommendation quality in real-world systems.

Abstract

The recent breakthrough of large language models (LLMs) in natural language processing has sparked exploration in recommendation systems, however, their limited domain-specific knowledge remains a critical bottleneck. Specifically, LLMs lack key pieces of information crucial for sequential recommendations, such as user behavior patterns. To address this critical gap, we propose IDLE-Adapter, a novel framework that integrates pre-trained ID embeddings, rich in domain-specific knowledge, into LLMs to improve recommendation accuracy. IDLE-Adapter acts as a bridge, transforming sparse user-item interaction data into dense, LLM-compatible representations through a Pre-trained ID Sequential Model, Dimensionality Alignment, Layer-wise Embedding Refinement, and Layer-wise Distribution Alignment. Furthermore, IDLE-Adapter demonstrates remarkable flexibility by seamlessly integrating ID embeddings from diverse ID-based sequential models and LLM architectures. Extensive experiments across various datasets demonstrate the superiority of IDLE-Adapter, achieving over 10\% and 20\% improvements in HitRate@5 and NDCG@5 metrics, respectively, compared to state-of-the-art methods.

Break the ID-Language Barrier: An Adaption Framework for LLM-based Sequential Recommendation

TL;DR

This work addresses the challenge of integrating domain-specific ID knowledge into LLM-based sequential recommendation. It introduces IDLE-Adapter, a two-component framework combining hard prompts and a four-part adapter that maps sparse ID interactions into dense LLM-compatible representations, with layer-wise refinement and distribution alignment guided by MMD. The approach yields significant gains over state-of-the-art ID-based and LLM-based methods across multiple datasets, while showing robust generalization across LLM backbones and ID models and providing interpretable bridging behavior. The proposed framework offers a practical pathway to leverage rich domain knowledge inside LLMs for improved recommendation quality in real-world systems.

Abstract

The recent breakthrough of large language models (LLMs) in natural language processing has sparked exploration in recommendation systems, however, their limited domain-specific knowledge remains a critical bottleneck. Specifically, LLMs lack key pieces of information crucial for sequential recommendations, such as user behavior patterns. To address this critical gap, we propose IDLE-Adapter, a novel framework that integrates pre-trained ID embeddings, rich in domain-specific knowledge, into LLMs to improve recommendation accuracy. IDLE-Adapter acts as a bridge, transforming sparse user-item interaction data into dense, LLM-compatible representations through a Pre-trained ID Sequential Model, Dimensionality Alignment, Layer-wise Embedding Refinement, and Layer-wise Distribution Alignment. Furthermore, IDLE-Adapter demonstrates remarkable flexibility by seamlessly integrating ID embeddings from diverse ID-based sequential models and LLM architectures. Extensive experiments across various datasets demonstrate the superiority of IDLE-Adapter, achieving over 10\% and 20\% improvements in HitRate@5 and NDCG@5 metrics, respectively, compared to state-of-the-art methods.

Paper Structure

This paper contains 27 sections, 14 equations, 4 figures, 4 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Sequential Recommendation
  4. LLMs for Recommendation
  5. Task Formulation
  6. Our Proposed Framework
  7. Hard Prompt Construction
  8. Adpater
  9. Pre-trained ID-based Sequential Model
  10. Dimensionality Alignment
  11. Layer-wise Embedding Refinement
  12. Layer-wise Distribution Alignment
  13. Joint Optimization
  14. Experiment
  15. Experimental Setup
  16. ...and 12 more sections

Figures (4)

  • Figure 1: The overall framework of IDLE-Adapter.
  • Figure 2: Generalizability evaluation of IDLE-Adapter across two series of language models, BGE and OPT. The performance of the second-best ID base model in Table \ref{['table:main']} is shown on the figure for reference.
  • Figure 3: Sensitivity of prompt length.
  • Figure 4: Visualization of the learned representations of the ID base model, the LLM and the adapter.