Temporal Interest Network for User Response Prediction

TL;DR

The paper addresses the lack of modeling semantic-temporal correlation between user history and target items in user response prediction. It introduces the Temporal Interest Network (TIN), which combines Target-aware Temporal Encoding (TTE), Target-aware Attention (TA), and Target-aware Representation (TR) to realize explicit 4-way interactions among behavior semantics, target semantics, behavior timing, and target timing. Empirical results on two public datasets show that TIN outperforms strong baselines in GAUC, with notable gains, and ablation confirms each component’s importance. The approach is validated in online A/B tests in Tencent's platform, achieving measurable cost and GMV lifts and has been deployed in production since 2023; code is released for replication. Overall, the work provides both a principled measurement of semantic-temporal correlation and a practical model that effectively leverages it to improve online advertising performance.

Abstract

User response prediction is essential in industrial recommendation systems, such as online display advertising. Among all the features in recommendation models, user behaviors are among the most critical. Many works have revealed that a user's behavior reflects her interest in the candidate item, owing to the semantic or temporal correlation between behaviors and the candidate. While the literature has individually examined each of these correlations, researchers have yet to analyze them in combination, that is, the semantic-temporal correlation. We empirically measure this correlation and observe intuitive yet robust patterns. We then examine several popular user interest models and find that, surprisingly, none of them learn such correlation well. To fill this gap, we propose a Temporal Interest Network (TIN) to capture the semantic-temporal correlation simultaneously between behaviors and the target. We achieve this by incorporating target-aware temporal encoding, in addition to semantic encoding, to represent behaviors and the target. Furthermore, we conduct explicit 4-way interaction by deploying target-aware attention and target-aware representation to capture both semantic and temporal correlation. We conduct comprehensive evaluations on two popular public datasets, and our proposed TIN outperforms the best-performing baselines by 0.43% and 0.29% on GAUC, respectively. During online A/B testing in Tencent's advertising platform, TIN achieves 1.65% cost lift and 1.93% GMV lift over the base model. It has been successfully deployed in production since October 2023, serving the WeChat Moments traffic. We have released our code at https://github.com/zhouxy1003/TIN.

Figures (7)

• Figure 1: (a): The ground truth Semantic-Temporal Correlation (STC) between history behaviors of Top-5 frequent categories (y-axis) at various target-relative positions from 1 to 10 (x-axis), and the target category of 674; (b,c,d): the learned STC in DIN, SASRec and BST. Numbers in parentheses denote the Pearson correlation coefficient between ground truth STC and learned STC on behaviors with the same category as the target (3rd row in each figure), i.e., 674.
• Figure 2: Left: the whole architecture of TIN; Right: the architecture of the Temporal Interest Module, which introduces target-aware temporal encoding, target-aware attention and representation to capture quadruple semantic-temporal correlations between behaviors and the target.
• Figure 3: Learned semantic-temporal correlation of TIN and its three ablated variants on the Amazon dataset. The numbers in parentheses represent the Pearson correlation coefficient between the ground truth (as shown in Fig. \ref{['subfig:674']}) and the learned correlation on behaviors with the same category as the target (3rd row).
• Figure 4: Time interval distribution and binning on Amazon and Alibaba dataset.
• Figure 5: Expressiveness of target-aware attention and target-aware representation in DIN. The green line denotes the performance of DIN with various attention embedding sizes $d_\text{TA}$, while the blue line denotes the performance of DIN with various representation embedding sizes $d_\text{TR}$. The red star corresponds to totally disabling the target-aware attention.

Theorems & Definitions (1)

• definition 1: Category-wise Target-aware Correlation (CTC)