Context-Aware Lifelong Sequential Modeling for Online Click-Through Rate Prediction

TL;DR

This work tackles click-through rate prediction under lifelong sequential modeling by introducing CAIN, a context-aware architecture that leverages Temporal Convolutional Networks to produce context-rich item representations. CAIN integrates a Multi-Scope Interest Aggregator to capture multiple context lengths and a Personalized Extractor Generation module to tailor convolutions to individual users. Across public and industrial datasets, CAIN consistently outperforms state-of-the-art lifelong sequential models and delivers notable online gains with modest latency overhead. The results suggest that context-aware, personalized sequence modeling can substantially improve recommendation quality in complex, real-world settings.

Abstract

Lifelong sequential modeling (LSM) is becoming increasingly critical in social media recommendation systems for predicting the click-through rate (CTR) of items presented to users. Central to this process is the attention mechanism, which extracts interest representations with respect to candidate items from the user sequence. Typically, attention mechanisms operate in a point-wise manner, focusing solely on the relevance of individual items in the sequence to the candidate item. In contrast, context-aware LSM aims to also consider adjacent items in the user behavior sequence to better assess the importance of each item. In this paper, we propose the Context-Aware Interest Network (CAIN), which utilizes the Temporal Convolutional Network (TCN) to create context-aware representations for each item throughout the lifelong sequence. These enhanced representations are then used in the attention mechanism instead of the original item representations to derive context-aware interest representations. Building upon this TCN framework, we propose the Multi-Scope Interest Aggregator (MSIA) module, which incorporates multiple TCN layers and their corresponding attention modules to capture interest representations across varying context scopes. Furthermore, we introduce the Personalized Extractor Generation (PEG) module, which generates convolution filters based on users' basic profile features. These personalized filters are then used in the TCN layers instead of the original global filters to generate more user-specific representations. We conducted extensive experiments on both a public dataset and an industrial dataset from the WeChat Channels platform. The results demonstrate that CAIN outperforms existing methods in terms of prediction accuracy and online performance metrics.

Figures (7)

• Figure 1: A comparison of attention scores from point-wise and context-aware perspectives.
• Figure 2: An overview of the proposed CAIN and its comparison to traditional LSM models. Figure (a) shows our baseline network that incorporates the LAP hou2024cross for LSM. Figure (b) shows an upgraded model that incorporates the TCN framework proposed in CAIN. Figure (c) is the final version of the proposed CAIN, which includes the MSIA and PEG modules.
• Figure 3: An illustration of the MSIA module. It contains several TCN layers and their corresponding attention modules to extract interest representations with respect to the candidate items within different context scopes.
• Figure 4: An illustration of the PEG module. It is a lightweight sub-network that takes the user's basic profile features as input and outputs the convolution filters that will be used in the TCN layers.
• Figure 5: Performance comparison with different context lengths.