Counterfactual Learning-Driven Representation Disentanglement for Search-Enhanced Recommendation

TL;DR

This work tackles negative transfer in search-enhanced recommendation by identifying a domain gap between search-specific intents and general item features. It introduces ClardRec, a counterfactual learning-driven framework that disentangles query-independent item features from query-related ones using a triplet counterfactual objective $L_{cd}$ and a constraint loss $L_{con}$, and transfers these features to recommendation via feature augmentation and data augmentation with a dedicated loss $L_{da}$. The method is implemented in an end-to-end pipeline with shared embeddings, domain-specific backbones, and a gated fusion mechanism, and is evaluated on KuaiSAR and E-commerce RS datasets across collaborative filtering and sequential settings, consistently outperforming baselines and showing robust gains even after ablations. The results support the effectiveness of separating general user-interest features from search-specific signals and demonstrate practical benefits for improving recommendation quality while mitigating negative transfer.

Abstract

For recommender systems in internet platforms, search activities provide additional insights into user interest through query-click interactions with items, and are thus widely used for enhancing personalized recommendation. However, these interacted items not only have transferable features matching users' interest helpful for the recommendation domain, but also have features related to users' unique intents in the search domain. Such domain gap of item features is neglected by most current search-enhanced recommendation methods. They directly incorporate these search behaviors into recommendation, and thus introduce partial negative transfer. To address this, we propose a Counterfactual learning-driven representation disentanglement framework for search-enhanced recommendation, based on the common belief that a user would click an item under a query not solely because of the item-query match but also due to the item's query-independent general features (e.g., color or style) that interest the user. These general features exclude the reflection of search-specific intents contained in queries, ensuring a pure match to users' underlying interest to complement recommendation. According to counterfactual thinking, how would user preferences and query match change for items if we removed their query-related features in search, we leverage search queries to construct counterfactual signals to disentangle item representations, isolating only query-independent general features. These representations subsequently enable feature augmentation and data augmentation for the recommendation scenario. Comprehensive experiments on real datasets demonstrate ClardRec is effective in both collaborative filtering and sequential recommendation scenarios.

Figures (7)

• Figure 1:
• Figure 2:
• Figure 3:
• Figure 5: Overview of ClardRec. "REC" and "SRC" denote the recommendation and search domains. The input of user historical sequences to the user backbone encoders in the sequential scenario is omitted for brevity.
• Figure 6: t-SNE visualization of query-independent and query-related item representations, user and item representations in recommendation using ClardRec across four backbones. We sampled 10,000 user-item click pairs for the collaborative filtering scenario and 1,000 user click sequences for the sequential scenario from the E-commerce R&S dataset. Stars indicate the distribution centers of representations.