Unifying Search and Recommendation in LLMs via Gradient Multi-Subspace Tuning

TL;DR

This work tackles the challenge of unifying search and recommendation within large language models by addressing gradient conflicts and knowledge drift that arise during parameter-efficient fine-tuning. It introduces Gradient Multi-Subspace Tuning (GEMS), which decomposes updates into a shared subspace and two task-specific subspaces to mitigate interference, and constrains updates with a null-space projection to preserve general-domain knowledge. Through extensive experiments on Amazon and Qilin datasets with multiple LLM backbones, GEMS consistently outperforms baselines and reduces both gradient conflicts and semantic drift, enabling efficient end-to-end S&R optimization. The approach promises practical benefits for deploying unified S&R systems in large-scale, real-world platforms by balancing specialization with knowledge retention and minimizing additional parameter overhead.

Abstract

Search and recommendation (S&R) are core to online platforms, addressing explicit intent through queries and modeling implicit intent from behaviors, respectively. Their complementary roles motivate a unified modeling paradigm. Early studies to unify S&R adopt shared encoders with task-specific heads, while recent efforts reframe item ranking in both S&R as conditional generation. The latter holds particular promise, enabling end-to-end optimization and leveraging the semantic understanding of LLMs. However, existing methods rely on full fine-tuning, which is computationally expensive and limits scalability. Parameter-efficient fine-tuning (PEFT) offers a more practical alternative but faces two critical challenges in unifying S&R: (1) gradient conflicts across tasks due to divergent optimization objectives, and (2) shifts in user intent understanding caused by overfitting to fine-tuning data, which distort general-domain knowledge and weaken LLM reasoning. To address the above issues, we propose Gradient Multi-Subspace Tuning (GEMS), a novel framework that unifies S&R with LLMs while alleviating gradient conflicts and preserving general-domain knowledge. GEMS introduces (1) \textbf{Multi-Subspace Decomposition}, which disentangles shared and task-specific optimization signals into complementary low-rank subspaces, thereby reducing destructive gradient interference, and (2) \textbf{Null-Space Projection}, which constrains parameter updates to a subspace orthogonal to the general-domain knowledge space, mitigating shifts in user intent understanding. Extensive experiments on benchmark datasets show that GEMS consistently outperforms the state-of-the-art baselines across both search and recommendation tasks, achieving superior effectiveness.

Figures (5)

• Figure 1: (a) Gradient conflict analysis across the layers of Qwen-3B under PEFT and our methods on the Qilin dataset. The Gradient Conflict Coefficient ($\rho$) du2018adapting quantifies the degree of opposition between the gradients of the S&R tasks (see Section \ref{['subsubsec:GCA']}); lower values indicate less conflict. The results show that our method consistently achieves lower gradient conflicts compared to PEFT. (b) Shifts in user intent understanding in real-world S&R.
• Figure 2: An overview of GEMS. During unified S&R training, task gradients are routed through multi-subspace decomposition and adaptively fused, then projected onto the null-space of pre-trained knowledge to preserve general-domain understanding.
• Figure 3: Ablation study of GEMS on Qilin.
• Figure 4: Gradient conflict heatmap analysis of PEFT and GEMS using Qwen-3B on the Qilin dataset.
• Figure 5: Hyperparameter analysis on Qilin.