Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

Eliminating Out-of-Domain Recommendations in LLM-based Recommender Systems: A Unified View

Hao Liao, Jiwei Zhang, Jianxun Lian, Wensheng Lu, Mingqi Wu, Shuo Wang, Yong Zhang, Yitian Huang, Mingyang Zhou, Rui Mao

TL;DR

This work introduces RecLM, a unified framework that eliminates out-of-domain item recommendations in LLM-based recommender systems by employing a shared <SOI>/<EOI> signaling mechanism and three grounding strategies: embedding-based retrieval, constrained generation with a Title Rewriter, and discrete item-tokenization. The three variants—RecLM-ret, RecLM-cgen, and RecLM-token—are trained under a common protocol and evaluated across three public datasets, achieving OOD@10 = 0 and, for the constrained-generation paths, state-of-the-art ranking performance. A key contribution is the systematic comparison of these paradigms under the same backbone model, revealing complementary strengths and trade-offs in accuracy, reliability, and efficiency. The framework also introduces lightweight enhancements such as a GRPO-trained Title Rewriter, scope-masked training, and multi-round dialogue training, and demonstrates practical viability with scalable prefix-tree decoding and cross-domain robustness. The results offer actionable guidance for deploying LLM-based recommender systems with reliable in-domain grounding, along with open-source code to facilitate replication and extension.

Abstract

Recommender systems based on Large Language Models (LLMs) are often plagued by hallucinations of out-of-domain (OOD) items. To address this, we propose RecLM, a unified framework that bridges the gap between retrieval and generation by instantiating three grounding paradigms under a single architecture: embedding-based retrieval, constrained generation over rewritten item titles, and discrete item-tokenizer generation. Using the same backbone LLM and prompts, we systematically compare these three views on public benchmarks. RecLM strictly eradicates OOD recommendations (OOD@10 = 0) across all variants, and the constrained generation variants RecLM-cgen and RecLM-token achieve overall state-of-the-art accuracy compared to both strong ID-based and LLM-based baselines. Our unified view provides a systematic basis for comparing three distinct paradigms to reduce item hallucinations, offering a practical framework to facilitate the application of LLMs to recommendation tasks. Source code is at https://github.com/microsoft/RecAI.

Eliminating Out-of-Domain Recommendations in LLM-based Recommender Systems: A Unified View

TL;DR

This work introduces RecLM, a unified framework that eliminates out-of-domain item recommendations in LLM-based recommender systems by employing a shared <SOI>/<EOI> signaling mechanism and three grounding strategies: embedding-based retrieval, constrained generation with a Title Rewriter, and discrete item-tokenization. The three variants—RecLM-ret, RecLM-cgen, and RecLM-token—are trained under a common protocol and evaluated across three public datasets, achieving OOD@10 = 0 and, for the constrained-generation paths, state-of-the-art ranking performance. A key contribution is the systematic comparison of these paradigms under the same backbone model, revealing complementary strengths and trade-offs in accuracy, reliability, and efficiency. The framework also introduces lightweight enhancements such as a GRPO-trained Title Rewriter, scope-masked training, and multi-round dialogue training, and demonstrates practical viability with scalable prefix-tree decoding and cross-domain robustness. The results offer actionable guidance for deploying LLM-based recommender systems with reliable in-domain grounding, along with open-source code to facilitate replication and extension.

Abstract

Recommender systems based on Large Language Models (LLMs) are often plagued by hallucinations of out-of-domain (OOD) items. To address this, we propose RecLM, a unified framework that bridges the gap between retrieval and generation by instantiating three grounding paradigms under a single architecture: embedding-based retrieval, constrained generation over rewritten item titles, and discrete item-tokenizer generation. Using the same backbone LLM and prompts, we systematically compare these three views on public benchmarks. RecLM strictly eradicates OOD recommendations (OOD@10 = 0) across all variants, and the constrained generation variants RecLM-cgen and RecLM-token achieve overall state-of-the-art accuracy compared to both strong ID-based and LLM-based baselines. Our unified view provides a systematic basis for comparing three distinct paradigms to reduce item hallucinations, offering a practical framework to facilitate the application of LLMs to recommendation tasks. Source code is at https://github.com/microsoft/RecAI.
Paper Structure (68 sections, 19 equations, 4 figures, 14 tables)

This paper contains 68 sections, 19 equations, 4 figures, 14 tables.

Table of Contents

  1. Introduction
  2. Related Work
  3. LLMs for Recommender Systems
  4. Addressing Out-of-domain Recommendations
  5. Methodology
  6. Special Item Indicator Token
  7. RecLM-ret
  8. RecLM-cgen
  9. Title Rewriter
  10. Recommendation-oriented Reward.
  11. Decoding Complexity.
  12. Conciseness.
  13. Discriminative Power.
  14. Scope Mask Training
  15. RecLM-token
  16. ...and 53 more sections

Figures (4)

  • Figure 1: An illustration of unconstrained versus constrained decoding.
  • Figure 2: Overview of RecLM variants: embedding-based retrieval (RecLM-ret), constrained generation over rewritten item titles (RecLM-cgen), and discrete item-tokenizer generation (RecLM-token).
  • Figure 3: Example implementation of RecLM-cgen during inference. Only minimal code modifications are required to integrate the constrained generation mechanism
  • Figure 4: Prompt templates used for the STR and GTR tasks. Variable placeholders are denoted in brackets.