Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Language Models as Semantic Augmenters for Sequential Recommenders

Mahsa Valizadeh, Xiangjue Dong, Rui Tuo, James Caverlee

TL;DR

The paper addresses the lack of semantic context in sequential recommender histories by introducing LaMAR, a framework that uses few-shot prompting of large language models to generate latent semantic signals from existing item metadata. These signals are integrated into sequential models through input-level augmentation and objective-aligned fine-tuning (including LoRA-based LM adaptation), yielding consistent performance gains across multiple datasets and base models. Human evaluation and diversity analyses indicate that the generated signals are relevant, informative, and sufficiently diverse to enrich representations. The work emphasizes a data-centric paradigm where LLMs act as intelligent context generators, enabling more nuanced recommendations and generalizing to new domains such as MovieLens.

Abstract

Large Language Models (LLMs) excel at capturing latent semantics and contextual relationships across diverse modalities. However, in modeling user behavior from sequential interaction data, performance often suffers when such semantic context is limited or absent. We introduce LaMAR, a LLM-driven semantic enrichment framework designed to enrich such sequences automatically. LaMAR leverages LLMs in a few-shot setting to generate auxiliary contextual signals by inferring latent semantic aspects of a user's intent and item relationships from existing metadata. These generated signals, such as inferred usage scenarios, item intents, or thematic summaries, augment the original sequences with greater contextual depth. We demonstrate the utility of this generated resource by integrating it into benchmark sequential modeling tasks, where it consistently improves performance. Further analysis shows that LLM-generated signals exhibit high semantic novelty and diversity, enhancing the representational capacity of the downstream models. This work represents a new data-centric paradigm where LLMs serve as intelligent context generators, contributing a new method for the semi-automatic creation of training data and language resources.

Language Models as Semantic Augmenters for Sequential Recommenders

TL;DR

The paper addresses the lack of semantic context in sequential recommender histories by introducing LaMAR, a framework that uses few-shot prompting of large language models to generate latent semantic signals from existing item metadata. These signals are integrated into sequential models through input-level augmentation and objective-aligned fine-tuning (including LoRA-based LM adaptation), yielding consistent performance gains across multiple datasets and base models. Human evaluation and diversity analyses indicate that the generated signals are relevant, informative, and sufficiently diverse to enrich representations. The work emphasizes a data-centric paradigm where LLMs act as intelligent context generators, enabling more nuanced recommendations and generalizing to new domains such as MovieLens.

Abstract

Large Language Models (LLMs) excel at capturing latent semantics and contextual relationships across diverse modalities. However, in modeling user behavior from sequential interaction data, performance often suffers when such semantic context is limited or absent. We introduce LaMAR, a LLM-driven semantic enrichment framework designed to enrich such sequences automatically. LaMAR leverages LLMs in a few-shot setting to generate auxiliary contextual signals by inferring latent semantic aspects of a user's intent and item relationships from existing metadata. These generated signals, such as inferred usage scenarios, item intents, or thematic summaries, augment the original sequences with greater contextual depth. We demonstrate the utility of this generated resource by integrating it into benchmark sequential modeling tasks, where it consistently improves performance. Further analysis shows that LLM-generated signals exhibit high semantic novelty and diversity, enhancing the representational capacity of the downstream models. This work represents a new data-centric paradigm where LLMs serve as intelligent context generators, contributing a new method for the semi-automatic creation of training data and language resources.
Paper Structure (23 sections, 4 equations, 7 figures, 9 tables)

This paper contains 23 sections, 4 equations, 7 figures, 9 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. Methodology of LaMAR
  4. Semantic Signal Generation via LLMs
  5. Signal Integration and Model Alignment
  6. Experiments
  7. Setup
  8. Semantic Signal Generation
  9. Can Semantic Signal Integration Enhance Sequential Recommenders?
  10. Can Semantic Signal Integration Improve LLM-Based Recommendation?
  11. Additional Results and Analysis
  12. Ablation on Prompts and Signal Numbers.
  13. Signal Diversity Analysis
  14. Generalization to a Different Domain
  15. Conclusion
  16. ...and 8 more sections

Figures (7)

  • Figure 1: Overview of the LaMAR framework. It consists of two stages: (1) semantic signal generation using a prompted large language model, and (2) integration of the generated signals into a sequential recommendation model via full fine-tuning, or into the LLM through parameter-efficient LoRA tuning.
  • Figure 2: Semantic similarity analysis across thresholds for signals generated by GPT-4o-mini and Gemini-1.5-Flash, using multi-qa-mpnet-base-cos-v1.
  • Figure 3: Semantic similarity analysis across thresholds for signals generated by GPT-4o-mini and Gemini-1.5-Flash, using all-MiniLM-L6-v2.
  • Figure 4: An example of prompt used for proposing a new relevant signal.
  • Figure 5: An example of prompt used for generating the proposed signal.
  • ...and 2 more figures