MLLMRec: A Preference Reasoning Paradigm with Graph Refinement for Multimodal Recommendation

TL;DR

MLLMRec tackles critical issues in multimodal recommendation by using a multimodal large language model to reason about purified user preference profiles and by refining the item-item graph with threshold-denoising and topology-aware strategies. The method generates semantic item descriptions from images, constructs behavior-driven user profiles, and fuses semantic and co-occurrence signals to learn high-order item representations via a LightGCN-like framework, optimized with a BPR objective. Empirical results on three public datasets show state-of-the-art performance with substantial gains, and ablation plus compatibility studies demonstrate robustness and transferability across MLLMs and baselines. The work offers a practical, plug-and-play approach to mitigate modality noise and graph structural noise, potentially accelerating deployment in real-world systems and guiding future research on reasoning-based recommendation and robust graph construction.

Abstract

Multimodal recommendation combines the user historical behaviors with the modal features of items to capture the tangible user preferences, presenting superior performance compared to the conventional ID-based recommender systems. However, existing methods still encounter two key problems in the representation learning of users and items, respectively: (1) the initialization of multimodal user representations is either agnostic to historical behaviors or contaminated by irrelevant modal noise, and (2) the widely used KNN-based item-item graph contains noisy edges with low similarities and lacks audience co-occurrence relationships. To address such issues, we propose MLLMRec, a novel preference reasoning paradigm with graph refinement for multimodal recommendation. Specifically, on the one hand, the item images are first converted into high-quality semantic descriptions using a multimodal large language model (MLLM), thereby bridging the semantic gap between visual and textual modalities. Then, we construct a behavioral description list for each user and feed it into the MLLM to reason about the purified user preference profiles that contain the latent interaction intents. On the other hand, we develop the threshold-controlled denoising and topology-aware enhancement strategies to refine the suboptimal item-item graph, thereby improving the accuracy of item representation learning. Extensive experiments on three publicly available datasets demonstrate that MLLMRec achieves the state-of-the-art performance with an average improvement of 21.48% over the optimal baselines. The source code is provided at https://github.com/Yuzhuo-Dang/MLLMRec.git.

Figures (6)

• Figure 1: Illustrations of (a) Noise interferes with user preference learning, (b) Distribution of similarity on the Baby dataset, and (c) Missing co-occurrence.
• Figure 2: The overall framework of MLLMRec. First, the MLLM transforms the images into the semantic descriptions, which are fused with the text to yield the multimodal descriptions. Next, MLLMRec constructs the behavioral description lists for users, which are fed into the MLLM to reason about the user preference profiles. Meanwhile, MLLMRec optimizes the item-item graph through two designed refinement strategies, subsequently learning high-order item representations on this refined graph.
• Figure 3: Effect of plugging the graph refinement strategies into other models on the Baby dataset.
• Figure 4: Performance of MLLMRec under different values of $\alpha$ and $K_{c}$ on three datasets.
• Figure 5: Performance under different values of $d$.