High Fidelity Textual User Representation over Heterogeneous Sources via Reinforcement Learning

TL;DR

This paper presents a reinforcement learning framework to synthesize compact, textual member representations from heterogeneous sources for large-scale, latency-sensitive recommender systems. It uses engagement signals as the primary reward and adds formatting and length constraints via auxiliary rewards, exploring pointwise and listwise reward formulations with entropy weighting. Through offline and online evaluations on LinkedIn products, the approach yields gains in predictive engagement and retrieval metrics, confirming the practicality and scalability of labeling-free textual representations that are directly compatible with LLM-based systems. The framework demonstrates robust performance across multiple signals, with careful design to mitigate reward hacking and maintain concise outputs, making it suitable for production deployment in multi-source personalization scenarios.

Abstract

Effective personalization on large-scale job platforms requires modeling members based on heterogeneous textual sources, including profiles, professional data, and search activity logs. As recommender systems increasingly adopt Large Language Models (LLMs), creating unified, interpretable, and concise representations from heterogeneous sources becomes critical, especially for latency-sensitive online environments. In this work, we propose a novel Reinforcement Learning (RL) framework to synthesize a unified textual representation for each member. Our approach leverages implicit user engagement signals (e.g., clicks, applies) as the primary reward to distill salient information. Additionally, the framework is complemented by rule-based rewards that enforce formatting and length constraints. Extensive offline experiments across multiple LinkedIn products, one of the world's largest job platforms, demonstrate significant improvements in key downstream business metrics. This work provides a practical, labeling-free, and scalable solution for constructing interpretable user representations that are directly compatible with LLM-based systems.

Figures (4)

• Figure 1: framework for member textual representation generation. The actor, a pre-trained finetuned via reinforcement learning, generates textual representations from member data, while the frozen reward model scores them based on their alignment with new job posting(s) (either pointwise or listwise prediction). The resulting reward signal is used to optimize the actor. When a member skips a recommended job, the job description is excluded to reduce input token length.
• Figure 2: Output token lengths are compared under three conditions: no constraint, explicit prompting, and using a length loss with a quadratic penalty function. Without any constraint, the output length grows rapidly, and we therefore stop the training process early. Explicit prompting substantially reduces this growth, although a slight upward trend persists and precise control over output length is not achieved. Applying the length penalty function stabilizes output length around $150$ tokens.
• Figure 3: Evolution of the reward score and the corresponding output token length on the training and validation datasets for pointwise and listwise tasks. The training curve is smoothed using a moving average with a window size of 50.
• Figure 4: Distribution of human ratings for content and accuracy (n=20)