TWIN V2: Scaling Ultra-Long User Behavior Sequence Modeling for Enhanced CTR Prediction at Kuaishou

TL;DR

TWIN-V2, an enhancement of TWIN, where a divide-and-conquer approach is applied to compress life-cycle behaviors and uncover more accurate and diverse user interests is introduced, which can compress behavior sequences well beyond the magnitude of 10^5 to a length manageable for online inference in GSU retrieval.

Abstract

The significance of modeling long-term user interests for CTR prediction tasks in large-scale recommendation systems is progressively gaining attention among researchers and practitioners. Existing work, such as SIM and TWIN, typically employs a two-stage approach to model long-term user behavior sequences for efficiency concerns. The first stage rapidly retrieves a subset of sequences related to the target item from a long sequence using a search-based mechanism namely the General Search Unit (GSU), while the second stage calculates the interest scores using the Exact Search Unit (ESU) on the retrieved results. Given the extensive length of user behavior sequences spanning the entire life cycle, potentially reaching up to 10^6 in scale, there is currently no effective solution for fully modeling such expansive user interests. To overcome this issue, we introduced TWIN-V2, an enhancement of TWIN, where a divide-and-conquer approach is applied to compress life-cycle behaviors and uncover more accurate and diverse user interests. Specifically, a hierarchical clustering method groups items with similar characteristics in life-cycle behaviors into a single cluster during the offline phase. By limiting the size of clusters, we can compress behavior sequences well beyond the magnitude of 10^5 to a length manageable for online inference in GSU retrieval. Cluster-aware target attention extracts comprehensive and multi-faceted long-term interests of users, thereby making the final recommendation results more accurate and diverse. Extensive offline experiments on a multi-billion-scale industrial dataset and online A/B tests have demonstrated the effectiveness of TWIN-V2. Under an efficient deployment framework, TWIN-V2 has been successfully deployed to the primary traffic that serves hundreds of millions of daily active users at Kuaishou.

Figures (5)

• Figure 1: The Proportion of user count and app usage duration at Kuaishou. We grouped users based on the number of videos they have viewed over the past three years. Highly active users, though a small percentage (12%), contribute the most usage duration (60%).
• Figure 2: Overview of TWIN-V2 in Kuaishou's CTR prediction system. Left: the offline component compresses life-cycle behaviors and extracts features of clustered behaviors. Right: the online component captures user interests from life-cycle behaviors, which adopts a two-stage approach. GSU and ESU for both stages take clustered behaviors as input.
• Figure 3: The deployment of TWIN-V2 at Kuaishou.
• Figure 4: Effects of core modules in TWIN-V2. Left: Comparisons between different hierarchical clustering methods. Right: Impact of omitting cluster size in the reweighting of relevance scores.
• Figure 5: GAUC improvement among users of three activity levels with different proportions. The improvement in GAUC is calculated by subtracting the GAUC of TWIN from the GAUC of TWIN-V2. TWIN-V2 demonstrates more improvements for users with a longer history.