Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

A Survey on Large Language Models in Multimodal Recommender Systems

Alejo Lopez-Avila, Jinhua Du

TL;DR

This survey investigates how large language models reshape multimodal recommender systems by focusing on prompting, training, and data adaptation strategies. It introduces an LLM-centric taxonomy that separates prompting, parameter-efficient tuning, and data-type adaptation, and situates these within core MRS tasks of disentanglement, alignment, and fusion. The work synthesises cross-domain transfer from related recommender domains, outlines comprehensive datasets and metrics, and identifies gaps and promising directions, including knowledge graph grounding, RAG-based reasoning, and agent-like LLM systems. Collectively, the findings illuminate how LLM capabilities can enable flexible, scalable, and interpretable multimodal recommendations with potential real-world impact across domains.

Abstract

Multimodal recommender systems (MRS) integrate heterogeneous user and item data, such as text, images, and structured information, to enhance recommendation performance. The emergence of large language models (LLMs) introduces new opportunities for MRS by enabling semantic reasoning, in-context learning, and dynamic input handling. Compared to earlier pre-trained language models (PLMs), LLMs offer greater flexibility and generalisation capabilities but also introduce challenges related to scalability and model accessibility. This survey presents a comprehensive review of recent work at the intersection of LLMs and MRS, focusing on prompting strategies, fine-tuning methods, and data adaptation techniques. We propose a novel taxonomy to characterise integration patterns, identify transferable techniques from related recommendation domains, provide an overview of evaluation metrics and datasets, and point to possible future directions. We aim to clarify the emerging role of LLMs in multimodal recommendation and support future research in this rapidly evolving field.

A Survey on Large Language Models in Multimodal Recommender Systems

TL;DR

This survey investigates how large language models reshape multimodal recommender systems by focusing on prompting, training, and data adaptation strategies. It introduces an LLM-centric taxonomy that separates prompting, parameter-efficient tuning, and data-type adaptation, and situates these within core MRS tasks of disentanglement, alignment, and fusion. The work synthesises cross-domain transfer from related recommender domains, outlines comprehensive datasets and metrics, and identifies gaps and promising directions, including knowledge graph grounding, RAG-based reasoning, and agent-like LLM systems. Collectively, the findings illuminate how LLM capabilities can enable flexible, scalable, and interpretable multimodal recommendations with potential real-world impact across domains.

Abstract

Multimodal recommender systems (MRS) integrate heterogeneous user and item data, such as text, images, and structured information, to enhance recommendation performance. The emergence of large language models (LLMs) introduces new opportunities for MRS by enabling semantic reasoning, in-context learning, and dynamic input handling. Compared to earlier pre-trained language models (PLMs), LLMs offer greater flexibility and generalisation capabilities but also introduce challenges related to scalability and model accessibility. This survey presents a comprehensive review of recent work at the intersection of LLMs and MRS, focusing on prompting strategies, fine-tuning methods, and data adaptation techniques. We propose a novel taxonomy to characterise integration patterns, identify transferable techniques from related recommendation domains, provide an overview of evaluation metrics and datasets, and point to possible future directions. We aim to clarify the emerging role of LLMs in multimodal recommendation and support future research in this rapidly evolving field.
Paper Structure (99 sections, 6 equations, 5 figures, 5 tables)

This paper contains 99 sections, 6 equations, 5 figures, 5 tables.

Table of Contents

  1. Introduction
  2. Strategy
  3. Differences among Other Surveys on MRS
  4. LLMs Beyond Encoders
  5. Taxonomy Based on LLM-Specific Functions
  6. Inclusion of Transferable Techniques from Related RS Domains
  7. Taxonomy
  8. Contributions of the Survey
  9. LLM methods
  10. Prompting
  11. Hard prompting
  12. Hard prompting
  13. Soft prompting
  14. Soft prompting
  15. Hybrid Techniques
  16. ...and 84 more sections

Figures (5)

  • Figure 1: Prompting strategies in LLM-based MRS.
  • Figure 2: Training strategies for adapting LLMs in Multimodal Recommender Systems.
  • Figure 3: Classification of modality adaptation strategies for integrating structured and multimodal inputs into LLMs.
  • Figure 4: Disentangle categories. Use-intent is not considered a category; it is just a note at the end of the section.
  • Figure 5: Alignment strategies.