Large Language Models for User Interest Journeys

TL;DR

This work demonstrates that large language models can reason through users' long-term activities to produce nuanced, human-readable descriptions of persistent 'journeys' in recommendation contexts. It introduces a two-part journey service: (1) Journey Extraction (ICPC) to derive coherent, personalized journey clusters from interaction histories, and (2) Journey Naming to generate descriptive, controllable names via prompt-tuned or fine-tuned LLMs. Through large-scale industrial data, the authors show that prompt-tuned naming on high-quality prompt data yields more accurate and interesting journey names, and that extraction is essential for high-quality naming. Early results suggest journey-aware recommendations can better serve users by aligning recommendations with their identified journeys. The study provides a roadmap for integrating LLM-based interpretability and user control into journey-aware recommender systems.

Abstract

Large language models (LLMs) have shown impressive capabilities in natural language understanding and generation. Their potential for deeper user understanding and improved personalized user experience on recommendation platforms is, however, largely untapped. This paper aims to address this gap. Recommender systems today capture users' interests through encoding their historical activities on the platforms. The generated user representations are hard to examine or interpret. On the other hand, if we were to ask people about interests they pursue in their life, they might talk about their hobbies, like I just started learning the ukulele, or their relaxation routines, e.g., I like to watch Saturday Night Live, or I want to plant a vertical garden. We argue, and demonstrate through extensive experiments, that LLMs as foundation models can reason through user activities, and describe their interests in nuanced and interesting ways, similar to how a human would. We define interest journeys as the persistent and overarching user interests, in other words, the non-transient ones. These are the interests that we believe will benefit most from the nuanced and personalized descriptions. We introduce a framework in which we first perform personalized extraction of interest journeys, and then summarize the extracted journeys via LLMs, using techniques like few-shot prompting, prompt-tuning and fine-tuning. Together, our results in prompting LLMs to name extracted user journeys in a large-scale industrial platform demonstrate great potential of these models in providing deeper, more interpretable, and controllable user understanding. We believe LLM powered user understanding can be a stepping stone to entirely new user experiences on recommendation platforms that are journey-aware, assistive, and enabling frictionless conversation down the line.

Figures (15)

• Figure 1: Our approach uses personalized clustering to uncover coherent user journeys, and names them via prompting LLMs.
• Figure 2: (left) Sample real journeys described by respondents. (right) Taxonomy of valued journeys uncovered by clustering text user responses via UMAP mcinnes2020umap.
• Figure 3: Infinite Concept Personalized Clustering (ICPC) on a User
• Figure 4: Visual depiction of an infinite concepts personalized clusters journey extraction for surfing. Shown at the top, the journey's salient terms representation. Below, the set of documents with a thumbnail and title for each. In contrast, multi-modal similarity topic clusters journeys only retrieve 6 documents, and co-occurrence topic clusters split these into 2 clusters, each with 2 documents.
• Figure 5: Comparison of journey extraction methods across proxy granularity metrics in (E1) setup.