Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

EncodeRec: An Embedding Backbone for Recommendation Systems

Guy Hadad, Neomi Rabaev, Bracha Shapira

TL;DR

This work tackles the mismatch between general-purpose pre-trained language models and the needs of recommender systems by introducing EncodeRec, a metadata-driven embedding backbone trained with a contrastive objective while keeping the language model parameters frozen. By aligning item titles with richer descriptions and attributes, EncodeRec produces domain-aware embeddings that improve both sequential recommendation and semantic ID tokenization tasks, outperforming strong baselines and reducing semantic ID collisions. The approach demonstrates scalable gains across multiple domains and backbone sizes, highlighting the value of embedding adaptation for bridging PLMs and practical recommendation systems. The results suggest promising future directions, including incorporating richer user signals and multi-modal item features to further enhance recommendation performance.

Abstract

Recent recommender systems increasingly leverage embeddings from large pre-trained language models (PLMs). However, such embeddings exhibit two key limitations: (1) PLMs are not explicitly optimized to produce structured and discriminative embedding spaces, and (2) their representations remain overly generic, often failing to capture the domain-specific semantics crucial for recommendation tasks. We present EncodeRec, an approach designed to align textual representations with recommendation objectives while learning compact, informative embeddings directly from item descriptions. EncodeRec keeps the language model parameters frozen during recommender system training, making it computationally efficient without sacrificing semantic fidelity. Experiments across core recommendation benchmarks demonstrate its effectiveness both as a backbone for sequential recommendation models and for semantic ID tokenization, showing substantial gains over PLM-based and embedding model baselines. These results underscore the pivotal role of embedding adaptation in bridging the gap between general-purpose language models and practical recommender systems.

EncodeRec: An Embedding Backbone for Recommendation Systems

TL;DR

This work tackles the mismatch between general-purpose pre-trained language models and the needs of recommender systems by introducing EncodeRec, a metadata-driven embedding backbone trained with a contrastive objective while keeping the language model parameters frozen. By aligning item titles with richer descriptions and attributes, EncodeRec produces domain-aware embeddings that improve both sequential recommendation and semantic ID tokenization tasks, outperforming strong baselines and reducing semantic ID collisions. The approach demonstrates scalable gains across multiple domains and backbone sizes, highlighting the value of embedding adaptation for bridging PLMs and practical recommendation systems. The results suggest promising future directions, including incorporating richer user signals and multi-modal item features to further enhance recommendation performance.

Abstract

Recent recommender systems increasingly leverage embeddings from large pre-trained language models (PLMs). However, such embeddings exhibit two key limitations: (1) PLMs are not explicitly optimized to produce structured and discriminative embedding spaces, and (2) their representations remain overly generic, often failing to capture the domain-specific semantics crucial for recommendation tasks. We present EncodeRec, an approach designed to align textual representations with recommendation objectives while learning compact, informative embeddings directly from item descriptions. EncodeRec keeps the language model parameters frozen during recommender system training, making it computationally efficient without sacrificing semantic fidelity. Experiments across core recommendation benchmarks demonstrate its effectiveness both as a backbone for sequential recommendation models and for semantic ID tokenization, showing substantial gains over PLM-based and embedding model baselines. These results underscore the pivotal role of embedding adaptation in bridging the gap between general-purpose language models and practical recommender systems.
Paper Structure (9 sections, 2 equations, 1 figure, 3 tables)

This paper contains 9 sections, 2 equations, 1 figure, 3 tables.

Table of Contents

  1. Introduction
  2. Related Works
  3. Text Embedding Models
  4. Text Encoders for Recommendation
  5. Methodology
  6. Experiments and Discussion
  7. Experimental Setup
  8. Results and Analysis
  9. Conclusion and Future Work

Figures (1)

  • Figure 1: Our metadata-contrastive training approach: concise semantic anchors (titles) are aligned with richer textual descriptions and attributes. Other items in the batch serve as negatives under the InfoNCE objective.