LIME: Link-based user-item Interaction Modeling with decoupled xor attention for Efficient test time scaling

TL;DR

The paper addresses the efficiency–expressiveness trade-off in large-scale recommender systems where history length $N(U)$ is large and candidate set size $M(U)$ is large, causing $O(N(U)^2)$ self-attention and linear-in-$M$ per-item complexity. It proposes LIME, which introduces a globally learned link-embedding sequence to decouple user history from candidate interactions and employs an XOR attention mask to achieve linear complexity in history length, with offline pre-computation of attention weights. Experiments on industrial and public data show LIME matches state-of-the-art transformers while delivering up to a 10x speedup for large candidate sets and long sequences, and deployment yields engagement gains with minimal online cost. The work also analyzes the low-rank structure of attention via SVD and demonstrates that a modest number of links (e.g., 32) captures most of the information needed for effective ranking, supporting the design choices and practical applicability of LIME.

Abstract

Scaling large recommendation systems requires advancing three major frontiers: processing longer user histories, expanding candidate sets, and increasing model capacity. While promising, transformers' computational cost scales quadratically with the user sequence length and linearly with the number of candidates. This trade-off makes it prohibitively expensive to expand candidate sets or increase sequence length at inference, despite the significant performance improvements. We introduce \textbf{LIME}, a novel architecture that resolves this trade-off. Through two key innovations, LIME fundamentally reduces computational complexity. First, low-rank ``link embeddings" enable pre-computation of attention weights by decoupling user and candidate interactions, making the inference cost nearly independent of candidate set size. Second, a linear attention mechanism, \textbf{LIME-XOR}, reduces the complexity with respect to user sequence length from quadratic ($O(N^2)$) to linear ($O(N)$). Experiments on public and industrial datasets show LIME achieves near-parity with state-of-the-art transformers but with a 10$\times$ inference speedup on large candidate sets or long sequence lengths. When tested on a major recommendation platform, LIME improved user engagement while maintaining minimal inference costs with respect to candidate set size and user history length, establishing a new paradigm for efficient and expressive recommendation systems.

Figures (10)

• Figure 1: Overall latency analysis across different model parameters. Both LIME models scale well with history length and number of candidates to rank whereas skyline model latencies explode. Note the logarithmic scale in the Latency axis.
• Figure 2: Architecture of TTSN (left), LIME-MHA (middle), and MHA (right). With a lightweight cross module using precomputed attention weights in a decoupled attention framework, LIME-MHA achieves MHA-level expressiveness with similar latency to TTSN.
• Figure 3: Architecture of HSTU Transformer (left) and LIME-XOR Decoupled Transformer (right) with a visual comparison of our proposed XOR attention mask against the standard HSTU causal self-attention mask.
• Figure 4: LIME Inference as part of the entire recommendation system
• Figure 5: Offline, online, and public dataset experimental results.