ID Embedding as Subtle Features of Content and Structure for Multimodal Recommendation

TL;DR

The paper investigates the underexplored semantics of item ID embeddings in multimodal recommendation, proposing that IDs encode subtle content- and structure-related signals. It introduces IDSF, a model that treats IDs as dual-purpose signals through modal-specific enhancements: a hierarchical attention-based content module augmented by ID embeddings and a lightweight, modality-specific structural module that aggregates neighborhood information. Content and structure representations are fused to form final item embeddings, trained with Bayesian Personalized Ranking alongside a multimodal contrastive objective, achieving state-of-the-art results on three real-world datasets and showing robustness to modality missing data. The work highlights the value of fine-grained ID usage for richer item representations and suggests future directions in sequential settings, user-side signals, and pre-training ID embeddings.

Abstract

Multimodal recommendation aims to model user and item representations comprehensively with the involvement of multimedia content for effective recommendations. Existing research has shown that it is beneficial for recommendation performance to combine (user- and item-) ID embeddings with multimodal salient features, indicating the value of IDs. However, there is a lack of a thorough analysis of the ID embeddings in terms of feature semantics in the literature. In this paper, we revisit the value of ID embeddings for multimodal recommendation and conduct a thorough study regarding its semantics, which we recognize as subtle features of \emph{content} and \emph{structure}. Based on our findings, we propose a novel recommendation model by incorporating ID embeddings to enhance the salient features of both content and structure. Specifically, we put forward a hierarchical attention mechanism to incorporate ID embeddings in modality fusing, coupled with contrastive learning, to enhance content representations. Meanwhile, we propose a lightweight graph convolution network for each modality to amalgamate neighborhood and ID embeddings for improving structural representations. Finally, the content and structure representations are combined to form the ultimate item embedding for recommendation. Extensive experiments on three real-world datasets (Baby, Sports, and Clothing) demonstrate the superiority of our method over state-of-the-art multimodal recommendation methods and the effectiveness of fine-grained ID embeddings. Our code is available at https://anonymous.4open.science/r/IDSF-code/.

Figures (8)

• Figure 1: An item consists of textual and visual modalities and ID information. Features in terms of content and structure can be enhanced in each modality separately. The item representation can be obtained by fusing content and structural features. Note that traditional ID embeddings are treated as content or structural features, while our approach takes ID embeddings as a mixture of subtle features of both content and structure.
• Figure 2: Distributions of pre-trained ID embeddings. The same color represents items that interacted with the same user.
• Figure 3: Semantic similarity of ID embeddings. Each cell in the heat map represents the normalized similarity of two ID embeddings, with horizontal and vertical coordinates denoting their mapped ID. Darker hues indicate a higher similarity.
• Figure 4: Semantic similarity of text and image features extracted by universal encoders. Each cell in the heat map represents the normalized similarity of two items, with horizontal and vertical coordinates denoting their mapped ID respectively. Darker hues indicate a higher similarity.
• Figure 5: Distributions of image and text features before and after the enhancement. Items interacted with the same user are represented by the same color.