Hierarchical Tree Search-based User Lifelong Behavior Modeling on Large Language Model

TL;DR

This work tackles the challenge of learning lifelong user interests from massive, long-horizon behavior data using large language models, whose context limits hinder direct lifelong modeling. It introduces HiT-LBM, a model-agnostic framework built from three components: Chunked User Behavior Extraction ($L$-based segmentation), Hierarchical Tree Search for Interests with process rating models ($SRM$ and $PRM$), and Temporal-Ware Interest Fusion to produce a dense, temporally aware lifelong user representation. The framework demonstrates substantial offline gains across multiple backbones and datasets, and online improvements in revenue and CVR on a real platform, including strong performance on long-tail users. The key contributions are (i) a cascading yet controlled chunk-based interest learning paradigm, (ii) a hierarchical search with learned continuity and effectiveness signals to identify optimal interest paths, and (iii) a temporal fusion mechanism that integrates long-horizon semantic representations into conventional recommendation pipelines, enabling practical deployment with improved performance and reduced hallucination risk.

Abstract

Large Language Models (LLMs) have garnered significant attention in Recommendation Systems (RS) due to their extensive world knowledge and robust reasoning capabilities. However, a critical challenge lies in enabling LLMs to effectively comprehend and extract insights from massive user behaviors. Current approaches that directly leverage LLMs for user interest learning face limitations in handling long sequential behaviors, effectively extracting interest, and applying interest in practical scenarios. To address these issues, we propose a Hierarchical Tree Search-based User Lifelong Behavior Modeling framework (HiT-LBM). HiT-LBM integrates Chunked User Behavior Extraction (CUBE) and Hierarchical Tree Search for Interest (HTS) to capture diverse interests and interest evolution of user. CUBE divides user lifelong behaviors into multiple chunks and learns the interest and interest evolution within each chunk in a cascading manner. HTS generates candidate interests through hierarchical expansion and searches for the optimal interest with process rating model to ensure information gain for each behavior chunk. Additionally, we design Temporal-Ware Interest Fusion (TIF) to integrate interests from multiple behavior chunks, constructing a comprehensive representation of user lifelong interests. The representation can be embedded into any recommendation model to enhance performance. Extensive experiments demonstrate the effectiveness of our approach, showing that it surpasses state-of-the-art methods.

Figures (7)

• Figure 1: Overview of the HiT-LBM framework.
• Figure 2: Example prompts for User Interest Learning and Item Knowledge Extraction, denoted as $prompt^{inst}$ and $prompt^{item}$ respectively.
• Figure 3: The prompts we construct for the recommendation task incorporate the user's previous interest ($prompt^{seq}$), current interest ($prompt^{point}$), and historical behavior ($prompt^{hist}$) respectively.
• Figure 4: The detailed workflow of how the Process Rating Model functions in hierarchical tree search.
• Figure 5: The impact of different modules in HiT-LBM on various backbone recommendation models.