MMP-Refer: Multimodal Path Retrieval-augmented LLMs For Explainable Recommendation

Abstract

Explainable recommendations help improve the transparency and credibility of recommendation systems, and play an important role in personalized recommendation scenarios. At present, methods for explainable recommendation based on large language models(LLMs) often consider introducing collaborative information to enhance the personalization and accuracy of the model, but ignore the multimodal information in the recommendation dataset; In addition, collaborative information needs to be aligned with the semantic space of LLM. Introducing collaborative signals through retrieval paths is a good choice, but most of the existing retrieval path collection schemes use the existing Explainable GNN algorithms. Although these methods are effective, they are relatively unexplainable and not be suitable for the recommendation field. To address the above challenges, we propose MMP-Refer, a framework using \textbf{M}ulti\textbf{M}odal Retrieval \textbf{P}aths with \textbf{Re}trieval-augmented LLM \textbf{F}or \textbf{E}xplainable \textbf{R}ecommendation. We use a sequential recommendation model based on joint residual coding to obtain multimodal embeddings, and design a heuristic search algorithm to obtain retrieval paths by multimodal embeddings; In the generation phase, we integrated a trainable lightweight collaborative adapter to map the graph encoding of interaction subgraphs to the semantic space of the LLM, as soft prompts to enhance the understanding of interaction information by the LLM. Extensive experiments have demonstrated the effectiveness of our approach. Codes and data are available at https://github.com/pxcstart/MMP-Refer.

Figures (7)

• Figure 1: Comparison of retrieval path collection methods between G-Refer and Ours(MMP-Refer).
• Figure 2: The overview framework of MMP-Refer.It mainly consists of three modules. Firstly, the multimodal representation learning module obtains the multimodal representations of users and items. Then, based on the multimodal features, rules are formulated and retrieval paths are collected through heuristic search; Finally, the textual features and collaborative features of the retrieval path are extracted as external knowledge to assist in fine-tuning LLM for generating high-quality recommendation explanations.
• Figure 3: Performance of different retrieved number
• Figure 4: Human evaluation comparing MMP-Refer with G-Refer
• Figure 5: Ablation study of Joint Residual Encoding on the sequential recommendation performance