Large Scale Retrieval for the LinkedIn Feed using Causal Language Models

TL;DR

This work tackles large-scale, low-latency retrieval for the LinkedIn Feed by fine-tuning a shared decoder-only LLM (LLaMA-3) into a dual-encoder that yields text-based embeddings for both members and items. Embeddings are generated via prompt-driven processing and pooled token representations, with cosine similarity guiding retrieval in a dense index, and trained using BCE and InfoNCE losses augmented by easy and hard negative mining and Matryoshka embeddings for multi-size representations. Offline evaluations and online A/B tests show that InfoNCE with Matryoshka learning substantially improves recall and overall engagement, particularly for newer members, while enabling storage-efficient deployment and sub-50 ms online retrieval at scale. The results demonstrate that generative language models can be adapted for industrial retrieval, achieving meaningful business gains through a unified, embedding-based retrieval stack and careful prompt encoding of count-based and historical features. Future work targets improved exploration, more efficient distillation to dual encoders, and further efficiency gains through shorter prompts and prompt-driven re-ranking layers.

Abstract

In large scale recommendation systems like the LinkedIn Feed, the retrieval stage is critical for narrowing hundreds of millions of potential candidates to a manageable subset for ranking. LinkedIn's Feed serves suggested content from outside of the member's network (based on the member's topical interests), where 2000 candidates are retrieved from a pool of hundreds of millions candidate with a latency budget of a few milliseconds and inbound QPS of several thousand per second. This paper presents a novel retrieval approach that fine-tunes a large causal language model (Meta's LLaMA 3) as a dual encoder to generate high quality embeddings for both users (members) and content (items), using only textual input. We describe the end to end pipeline, including prompt design for embedding generation, techniques for fine-tuning at LinkedIn's scale, and infrastructure for low latency, cost effective online serving. We share our findings on how quantizing numerical features in the prompt enables the information to get properly encoded in the embedding, facilitating greater alignment between the retrieval and ranking layer. The system was evaluated using offline metrics and an online A/B test, which showed substantial improvements in member engagement. We observed significant gains among newer members, who often lack strong network connections, indicating that high-quality suggested content aids retention. This work demonstrates how generative language models can be effectively adapted for real time, high throughput retrieval in industrial applications.

Figures (4)

• Figure 1: Architecture of the current Suggested Content Ranking and Retrieval Set up. Five sources are shown as examples to illustrate retrieval from multiple sources. In practice, the system has around many more sources
• Figure 2: Dual encoder architecture utilizing a shared LLM for text-based retrieval. Member and item texts are processed separately through the LLM, generating token-level hidden representations. A pooling layer aggregates these representations into embeddings, which are then compared using a similarity function.
• Figure 3: Online System for the Retrieval and Ranking Model
• Figure 4: Results for Recall@10 versus Embedding Dimension when training using MRL.