From Token to Item: Enhancing Large Language Models for Recommendation via Item-aware Attention Mechanism

Abstract

Large Language Models (LLMs) have recently gained increasing attention in the field of recommendation. Existing LLM-based methods typically represent items as token sequences, and apply attention layers on these tokens to generate recommendations. However, by inheriting the standard attention mechanism, these methods focus on modeling token-level relations. This token-centric focus overlooks the item as the fundamental unit of recommendation, preventing existing methods from effectively capturing collaborative relations at the item level. In this work, we revisit the role of tokens in LLM-driven recommendation and categorize their relations into two types: (1) intra-item token relations, which present the content semantics of an item, e.g., name, color, and size; and (2) inter-item token relations, which encode collaborative relations across items. Building on these insights, we propose a novel framework with an item-aware attention mechanism (IAM) to enhance LLMs for recommendation. Specifically, IAM devises two complementary attention layers: (1) an intra-item attention layer, which restricts attention to tokens within the same item, modeling item content semantics; and (2) an inter-item attention layer, which attends exclusively to token relations across items, capturing item collaborative relations. Through this stacked design, IAM explicitly emphasizes items as the fundamental units in recommendation, enabling LLMs to effectively exploit item-level collaborative relations. Extensive experiments on several public datasets demonstrate the effectiveness of IAM in enhancing LLMs for personalized recommendation.

Figures (6)

• Figure 1: (a) Current LLM-based methods focus on modeling token-level relations, while failing to exploit collaborative relations at the item level; (b) intra- and inter-item token relations indicate item content semantics and collaborative relations, respectively.
• Figure 2: Recommendation paradigm of LLM-based methods.
• Figure 3: Causal self-attention mechanism and its attention matrix.
• Figure 4: The architecture of IAM. The intra-item attention layer handles intra-item token relations to model item content semantics. The inter-item attention layer attends to inter-item token relations to explicitly capture item collaborative relations. IAM employs a recommendation adapter to generate top-$k$ items as recommendations.
• Figure 5: Ablation study on intra- and inter-item attention layers.